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16  Abstract 

The  purpose  of  this  document  is  to  identify  various  software  reliability 
models,  define  the  interface  between  a  software  reliability  model  with 
a  fault  tolerant  system  reliability  model,  and  provide  a  software  depend¬ 
ability  model  (capable  of  evaluating  availability,  reliability,  and  safety) 
that  can  predict  the  reliability  of  software  prior  to  and  throughout  its 
development.  The  software  reliability  data  and  development  of  a  software 
reliability  data  base  is  also  discussed.  ,  ’ 
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PREFACE 

The  dependability  property  of  a  computer  system  allows  reliance 
to  be  justifiably  placed  on  the  service  it  delivers,  which  is  its  behavior 
as  perceived  by  its  users  [AVLA86].  This  concept  naturally  encompasses 
the  notions  of  reliability,  availability,  and  safety. 

The  purpose  of  dependability  is  the  design,  implementation, 
and  use  of  computer  systems  where  faults  are  natural,  foreseeable,  and 
tolerable  [T0UL76]. 

Or.  John  P.  J.  Kelly 
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TECHNICAL  REPORT 


on 

SOFTWARE  DEPENDABILITY  ASSESSMENT  METHODS 

1.0  INTRODUCTION 

The  increasing  application  and  criticality  of  digitally  implemented 
flight  control  functions  dictates  a  need  for  highly  dependable  software. 

A  variety  of  methods  for  creating  reliable  software  have  been  developed  (e.g., 
software  engineering,  higher  order  languages,  testing  and  debugging  procedures). 
These  methods  do  not  inherently  provide  a  measure  of  reliability  improvement 
obtained  using  the  methods.  Methods  of  assessing  the  reliability  and  depend¬ 
ability  of  software  are  needed.  This  effort  was  originally  entitled  "software 
reliability  assessment  methods"  but  as  the  work  progressed,  Battelle  and 
our  consultant.  Dr.  John  P.  J.  Kelly,  determined  that  systems  which  provide 
functions  critical  to  the  safe  transportation  of  passengers  must  be  more 
than  reliable,  they  must  also  be  dependable.  As  stated  in  the  preface  written 
by  Dr.  Kelly,  the  concept  of  dependability  encompasses  the  notions  of  reliability, 
availability,  and  safety. 

The  overall  objective  of  this  research  was  to  investigate  methods 
of  assessing  the  reliability  of  digtial  avionics  software  developed  using 
primarily  higher  order  languages. 

2.0  BACKGROUND 

Many  software  reliability  models  have  been  developed  which  predict 
the  reliability  of  software  based  on  various  input  parameters.  That  work 
was  sponsored  by  the  USAF  Rome  Air  Development  Center  (RADC)  and  NASA's  Ames 
and  Langley  Research  Centers.  Many  of  these  studies  attempted  to  analyze 
an  existing  data  base  to  derive  a  prediction  methodology.  Many  of  these 
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data  bases  were  files  of  historical  significance,  and  were  not  real-time 
airborne  software  systems  developed  using  higher  order  languages.  Consequently 
many  researchers  have  found  the  popular  software  reliability  models  are  invalid 
Recent  work  involved  carful ly  designed  and  controlled  software  development 
experiments  using  a  limited  number  of  programmers  programming  a  limited  number 
of  problems.  That  work,  in  turn,  has  been  criticized  as  not  representative 
of  real-time  digital  avionics  software. 

3.0  SUMMARY 

This  report  is  in  five  sections.  The  following  four  sections  present 

(1)  a  summary  of  the  results  of  previous  studies  of  software 
reliability  models  applicable  to  real-time  software 

(2)  a  definition  of  the  software  dependability  model  interfaces 
with  fault  tolerant  system  reliability  models 

(3)  formulation  of  a  software  dependability  model 

(4)  a  definition  of  software  data  to  be  collected  by  the  avionics 
system  developer  for  storage  in  the  software  dependability 
data  base. 
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TECHNICAL  REPORT 

on 

REVIEW  OF  PREVIOUS  STUDIES  OF 
SOFTWARE  RELIABILITY  MODELS 

1.0  INTRODUCTION 

Several  studies  of  software  reliability  models  have  been  reviewed. 

In  addition  to  those  models  covered  in  the  "Comparative  Analysis  of  Fault 
Tolerant  Software  Design  Techniques",  prepared  by  Battelle  Columbus  Division 
on  February  15,  1984,  other  models  have  been  reviewed  through  the  use  of 
published  technical  papers  and  a  software  engineering  textbook. (1)  The 
technical  papers  are  listed  in  Table  1  and  divided  into  the  following  cate¬ 
gories:  "For  Information  Only",  "Discusses  Reliability  Modeling  Applicable 
to  Real-Time  Software",  and  "Discusses  Reliability  Modeling  Not  Applicable 
to  Real-Time  Software".  Table  2  gives  the  names  of  the  reliability  models 
and  indicates  whether  or  not  they  are  applicable  to  real-time  software. 

Only  the  software  reliability  models  which  are  applicable  to  real-time  software 
will  be  discussed  in  further  detail. 

The  following  software  reliability  models  have  the  character!' sti c 
of  being  applicable  to  real-time  software.  The  criteria  used  to  determine 
real-time  applicability  was: 

(a)  The  model  is  not  extremely  difficult  to  solve  numerically. 

Hence,  only  a  reasonable  amount  of  computations  is  required. 

(b)  The  model  must  take  into  account  the  test  time. 


(!)  Martin  L.  Shooman 

SOFTWARE  ENGINEERING  Desiqn/Rel iabi 1 i ty/Manaqement , 
McGraw-Hill  Book  Company,  New  York,  1983. 
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Geneva. ieed  ‘Lrnpi.-'fs.o-c  Debupaina  rvt  nda.! 

Stoonc-i'^a  Model 

8uQ  •  Pr 3p optional  Model 

Pi'aar-na.t'.c.  iortwant  Reliability  Prediction  Model 
by  Wall  ana  Ferguson 

GeneraLixta  Poisson  Model  by  Weibu.Il  (.WogonerG 
Geometric  PoiS3en  Model  by  rvLoranda. 

Gael-  OKumoto  Non  -  Homogeneous  Poisson  M.cdel 
Schneidewird  Won- Homogeneous  Poisson  Model 
Goel  Modified  (Von- Homogeneous  Poisson  Model 

Green  Mode! 

yVeibuU  L  Coudinho')  Model 
Corcoran- Wemaar  ten- lebna.  Model 
VY  ciSS  Model 

Mu-Sa.  Execution  Tims  Failure  Motel 
JelmsKi  -  Nlcranba  Ce-EutropHicatr.cn  Model 
Extended  J»)  mSM- Moroj-ca  Model 

Geometric  De  -  eutrophication  Modei  by  Mor  an  eta. 

Modified  Geometric  De- Eutrophication  Mode.1 
Us  Rad  lula  Reliability  Growth  Model 


x 


X 


X 
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X 


X 

X 

X 

X 


TABLE  2.  REVIEW  SOFTWARE  RELIABILITY  MODELS 


Serf  ■twcu'e.  Reliability  Made.! 


Schick  -  V\'c\v£<  ten  Linear  Model 

£x-ttndib  Sen  iCh- Wolver  ton  Mooel  by  Lipow 
Modified  Schuck-  Vv'ol  vtr  ten  Mead  by  Su.hert 
S'nooman  Struct ur cii  Model 
Snooman  Exponential  Model 

Shiooman "  Natarajan  Manpower  Limited  Model 
Miyamoto"  s  Revised  Shoenan  Model 
Markov  Model  by  Shoe  man  and  Trivedl 
Littlewood  Decreasing  failure  it  Modil 
Little  wood  Markov  Model 

Sayesian  Reliability  Growth  M-ockl  by  t^^Verroll 


*  Reasons: 

a.  The  model  is  extremely  difficult  to  solve  numerically.  Consequently,  the 
variables  must  be  selected  so  as  to  limit  the  amount  of  computation  required. 

b.  The  test  time  is  totally  ignored. 

c.  The  technique(s)  used  is  (are)  undesirable  in  software  reliability  models. 

d.  The  model  does  not  adequately  address  the  overall  software  reliability  problem 

e.  There  are  massive  difficulties  in  estimating  the  parameters. 

f.  Some  of  the  assumptions  are  quite  questionable. 


TABLE  2.  REVIEWED  SOFTWARE  RELIABILITY  MODELS  (Continued) 
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(c)  The  techniques  used  in  the  model  must  be  reasonable.  (For 
example,  the  cumulative  averaging  technique  is  undesirable 
in  software  reliability  models  because  this  technique  causes 
early  data  points  to  be  weighted  more  heavily  than  later  ones. 
Thus,  with  this  technique,  even  the  most  erratic  error  data 
will  eventually  project  a  "fitted  line"  as  the  sample  size 
becomes  sufficiently  large). 

(d)  The  model  addresses  both  the  dynamic  and  static  measurements: 

(1)  Dynamic  measurements 

•  hazard  rate,  z(t) 

•  reliability  function,  R(t) 

•  mean  time  to  failure,  MTTF 

(2)  Static  measurements 

•  total  number  of  errors 

•  total  number  of  remaining  errors 

(e)  The  parameters  are  not  diffcult  to  estimate. 

(f)  The  assumptions  are  reasonable. 


The  underlying  assumptions,  key  features,  and  study  results  are 
summarized  below. 

2.0  GENERALIZED  IMPERFECT  DEBUGGING  MODEL 
2.1  Underlying  Assumptions 

a.  All  failures  are  observable  and  independent. 

b.  The  time  to  remove  a  failure  is  considered  to  be  negligible 
and  is  ignored  in  the  model, 

c.  Errors  are  not  always  corrected  when  detected  and  errors  may 
be  spawned  when  correcting  errors. 

d.  Testing  is  of  uniform  intensity  and  representative  of  the 
operational  environment. 

e.  Inputs  which  exercise  the  program  are  randomly  selected. 

f.  The  failure  rate  at  any  time  is  proportional  to  the  current 
number  of  errors  remaining  in  the  program. 

g.  The  failure  rate  between  the  (i-l)th  failure  and  the  ith  failure 
is  x ( t i )  =  o[N-(1-l)]t— 1. 
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2.2  Key  Features 

The  failure  rate  is  of  the  form 

X(t)  =  $ (N-p(i-l)  )t““l 

with  <p  =  a  proporti onal  i ty  constant; 

N  =  the  total  number  of  errors; 

p  =  the  probability  of  perfect  programmer  debugging  behavior 
*  =  the  parameter  that  controls  the  shape  of  the  failure  rate 


The  reliability  functions  is 


R(t)  =  exp  (- 


t>  (N-p(i-l) ) 


t“) 


and  the  Mean  Time  to  Failure  is 


MTTF 


2.3  Study  Results 


(N-p(i-l) ) 


r  (4-). 


This  model  cannot  determine  the  effect  each  bug  contributes  to 
the  overall  failure  rate  without  continuing  to  run  the  program  because  the 
instantaneous  failure  rate  will  be  zero  when  a  bug  is  found  and  immediately 
removed.  The  model  provides  a  good  fit  with  data.  The  parameter  estimates 
are  reasonable  for  the  data  sets  tested. 


3.0  BUG-PROPORTIONAL  MODEL 


3.1  Underlying  Assumptions 


a.  The  number  of  errors  in  a  program  is  a  constant  and  decreases 
directly  as  errors  are  corrected. 

b.  Software  errors  are  caused  by  the  uncovering  of  residual  bugs 
in  a  program. 


c.  The  probability  that  a  bug  is  encountered  in  the  time  interval. 
At,  after  t  successful  hours  of  operation  is  proportional  to 
the  fractional  number  of  remaining  bugs. 

d.  The  fractional  number  of  remaining  bugs  is  independent  of  the 
operating  time. 

e.  The  rate  of  error  correction  is  constant. 


3.2  Key  Features 


The  hazard  rate  is  of  the  form 


z(t)  «  Ker(t)  =  K[(Et/Ij)  -  ec(t)] 


where  K  =  an  arbitrary  constant;  and 
er( t)  =  the  number  of  remaining  bugs; 

Ej  =  the  total  number  of  errors  originally  present; 
Ij  =  the  total  number  of  machine  instructions;  and 
£c(t)  *  the  number  of  corrected  bugs. 


The  reliability  function  is 


R(t)  =  exp  {— [ Ke r ( T ) 3 1 }  =  exp  f -K[( Ej/It)  -  EC(T)]t; 


and  the  Mean  Time  to  Failure  is 


MTTF 


1 


Ker(T  ) 


- — r—  with  6  =  ILk  and  «  =  .^II, 

b(i-*t)  it  et 


where  3g  =  a  constant  rate  of  error  correction. 


3.3  Study  Results 


The  overall  behavior  of  the  model  is  verified.  However,  the  errors 
between  measurement  and  prediction  had  a  standard  deviation  of  24  percent. 
When  seeking  historical  data  for  estimation  of  reliability  parameters,  the 
examples  should  closely  match  the  intended  application  and  phase. 
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4.0  GEOMETRIC  POISSON  MODEL 


4.1  Underlying  Assumptions 


a.  There  is  an  infinite  number  of  errors. 

b.  Each  fault  in  the  program  is  independent  of  the  others  and 
each  of  them  is  equally  likely  to  occur. 

c.  The  errors  do  not  have  the  same  likelihood  of  detection. 

d.  During  a  fixed  interval  of  time,  the  number  of  errors  detected 
follows  a  Poisson  distribution. 

e.  During  each  of  these  periods  of  time,  the  detection  rate  is 
constant. 

f.  Data  is  available  only  at  discrete  intervals. 

g.  The  detection  rate  in  successive  time  intervals  forms  a  geometric 
progression. 

h.  Each  error  discovered  is  immediately  removed  or  no  longer  counted 

i.  No  new  fault  is  introduced  during  a  correction  time. 


4.2  Key  Features 


The  hazard  rate  during  the  ith  time  interval  is 


z(tj)  =  *10-1 


where  tj  =  the  ith  debugging  interval; 

*  =  the  average  number  of  faults  occurring  in  the  first 
i nterva 1 ; 

K  =  a  proporti onal i ty  constant,  0  <  K  <  1. 


The  reliability  function  is 


and  the  Mean  Time  to  Failure  is 


MTTF  =  — L 

XK1' 

4.3  Study  Results 


This  model  gives  identical  results  as  the  Schneidewind  Non-Homogeneous 

model . 


5.0  SCHNEIDEWIND  NON-HOMOGENEOUS  POISSON  MODEL 


5.1  Underlying  Assumptions 


a.  The  number  of  errors  which  is  detected  during  a  time  interval 

and  the  collection  of  error  counts  over  a  series  of  time  intervals 
are  modelled  by  a  random  variable  and  a  stochastic  process. 

b.  Prior  to  the  selection  of  a  test  plan,  all  errors  are  equally 
1 i kely. 

c.  The  number  of  errors  detected  in  each  time  interval  is  inde¬ 
pendent  of  the  number  detected  in  another  time  interval. 

d.  Detected  error  counts  in  each  interval  have  the  same  type  of 
distribution  but  have  different  means. 

e.  The  mean  number  of  detected  errors  decreases  from  interval 
to  interval. 

f.  The  rate  of  detection  in  an  interval  is  proportional  to  the 
number  of  errors  in  that  interval. 

g.  The  error  process  is  a  non-homogeneous  Poisson  process  with 
an  exponentially  decreasing  intensity  function. 

h.  The  error  correction  rate  is  proportional  to  the  number  of 
errors  to  be  corrected. 


5.2  Key  Features 


& 

A 


u 


The  hazard  function  is  equivalent  to  the  predicted  number  of  errors 
for  each  interval  i  where 
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m-j  =  (°73)[exp  (-B(i-l))  -  exp  (-Si)] 

with  m-j  =  the  estimated  number  of  errors  in  interval  i; 

8  =  a  model  constant;  and 

1  =  a  model  constant  (error  detection  rate  at  time  0). 

The  weighted  squared  deviation  is 
t 

SDW  =  I  exp  ( Si )  [(*/B)  [exp  (-3i)][exp  (S)-l]  -X-j]2. 
k=l 

5.3  Study  Results 

This  model  is  equivalent  to  the  Geometric  De-Eutrophication  model. 
However,  this  model  offers  greater  flexibility  than  the  Geometric  De-Eutro¬ 
phication  model . 

The  required  test  data  for  this  model  consists  of  the  sequence 
of  the  number  of  errors  in  each  time  interval. 

6.0  JELINSKI-MORANDA  DE-EUTROPHICATION  MODEL 

6.1  Underlying  Assumptions 

a.  A  program  can  be  decomposed  into  a  number  of  paths  or  cases. 

b.  The  identification  of  paths  will  be  done  at  a  high  enough  level 
to  yield  a  relatively  small  number  of  cases  (jclO2^). 

c.  The  number  of  machine  language  instructions  remains  relatively 
constant. 

d.  Failure  is  caused  by  rare  combinations  of  input  data  and  path 
traversals,  with  the  time  between  failures  governed  by  an 
exponential  distribution,  yielding  a  constant  hazard. 

e.  There  is  a  fixed  number  of  errors  in  the  program. 

f.  No  new  errors  are  added  during  the  debugging  process. 

g.  Each  error  discovered  is  immediately  removed. 

h.  Each  error  has  an  equal  chance  of  being  detected. 


The  failure  rate  is  proportional  to  the  current  error  content 
(number  of  remaining  errors). 

The  program  is  not  being  altered  except  for  error  correction. 
Only  one  error  may  occur  in  a  given  time  debugging  period. 


6.2  Key  Features 


The  hazard  function  is  of  the  form 


Z ( X -j )  =  *  [N-(i-l)] 


with  N  = 


=  the  total  number  of  initial  errors  in  the  program; 

=  a  proportionality  constant; 

=  the  length  of  the  ith  debugging  interval  (the  time  between 
detection  of  the  (i-l)st  and  the  ith  errors);  and 

=  the  number  of  errors  discovered. 


The  reliability  function  is 


R(Xi )  =  exp  [-<j(N-n)X-j] 


and  the  Mean  Time  to  Failure  is 

MTTF  =  l/0(N-n)] 

where  n  =  the  number  of  errors  found  to  date. 

6.3  Study  Results 

The  model  provides  a  good  fit  with  data.  The  model  runs  into  slight 
trouble  with  its  "no  new  errors"  assumption.  At  the  "last"  error,  the  time 
between  errors  shows  a  sudden  sharp  increase  which  is  somewhat  optimistic. 

The  larger  the  data  set,  the  more  likely  the  sudden  improvement  and  the  impli¬ 
cation  that  debugging  is  complete. 


_V  ■. ■  V.V*V J.%  A  . V .y.VA.S  .N  N.V.V 

«■*_  /.  /.  /«  «.  A  A  *  .A/-  ’ 
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This  model  requires  a  sequence  of  times  between  failures  in  order 
to  estimate  the  parameters. 

7.0  EXTENDED  JELINSKI-MQRANDA  MODEL 

7.1  Underlying  Assumptions 


a.  There  is  a  fixed  number  of  errors  in  the  program. 

b.  No  new  errors  are  added  during  the  debugging  process. 

c.  Each  error  discovered  is  immediately  removed. 

d.  Each  error  has  an  equal  chance  of  being  detected. 

e.  There  is  a  constant  failure  rate  between  consecutive  errors. 

f.  A  program  can  be  decomposed  into  a  number  of  paths  or  cases. 

g.  The  identification  of  paths  will  be  done  at  a  high  enough  level 
to  yield  a  relatively  small  number  of  cases  (<10^0) . 

h.  The  number  of  machine  language  instructions  remains  rel  tively 
constant. 

i.  The  failure  rate  is  proportional  to  the  current  error  content 
(number  of  remaining  errors). 

j.  The  program  is  not  being  altered  except  for  error  correction. 

k.  More  than  one  error  may  occur  in  a  given  time  debugging  period. 


7.2  Key  Features 


The  hazard  function  is  of  the  form 


z ( t-j )  =  <p  [N-n-j  _  x] 


where  h  = 


a  proportional i ty  constant; 

the  total  number  of  initial  errors; 

the  cumulative  number  of  errors  found  through  the  ith 
time  i nterval ;  and 

the  ith  debugging  interval. 


.  -»  ^  V  «•  / 


-  •  *\  •'V  .  -  .  «  -  •  *  *  '■>  '  •  •  • 


2-13 


The  reliability  function  is 

R ( t )  =  e  "  f  (fJ_ni  )t 
and  the  Mean  Time  to  Failure  is 


MTTF  =  - i —  . 

0  [  N  -  n  -,•  ] 


7.3  Study  Results 

This  model  requires  the  number  of  errors  in  some  uniform  time  period 
to  estimate  the  parameters. 

The  model  provides  a  good  fit  with  data.  The  model  is  somewhat 
inconsistent  and  less  smooth  due  to  its  use  of  actual  errors  in  its  hazard 
function.  Fairly  small  changes  in  the  data  give  a  significant  change  in 
the  model's  shape  and  prediction.  At  the  "last"  error,  the  time  between 
errors  shows  a  sudden  sharp  increase  which  is  somewhat  optimistic. 


8.0  GEOMETRIC  DE-EUTROPHICATION  MODEL 


8.1  Underlying  Assumptions 


a.  There  is  an  infinite  number  of  total  errors. 

b.  Each  fault  in  the  program  is  independent  of  the  others  and 
each  of  them  is  equally  likely  to  occur. 

c.  The  errors  do  not  have  the  same  likelihood  of  detection. 

d.  Each  error  discovered  is  immediately  removed.  The  time  to 
correct  the  detected  faults  is  negligible. 

e.  No  new  fault  is  introduced  during  the  correction  time. 

f.  The  failure  rate  between  successive  errors  forms  a  geometric 
progression  and  is  constant  in  the  interval  between  errors. 
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8.2  Key  Features 


The  hazard  function  is 


Z(Xj)  =  DK1’! 


1 


where  X-j  =  the  ith  debugging  interval; 

D  =  the  initial  error  detection  rate; 

K  =  a  proportionality  constant;  and 
i  =  the  number  of  errors  discovered  after  i  intervals. 

The  reliability  function  is 

R( X-j )  =  exp[-DKnXj  ] 

where  n  =  the  total  number  of  errors  discovered 


and  the  Mean  Time  to  Failure  is 


MTTF  = 


8.3  Study  Results 

The  test  data  necessary  to  apply  this  model  is  the  sequence  of 
times  between  errors. 

This  model  gives  a  reasonable  fit  with  data.  It  is  also  fairly 
consistent  in  its  results  when  only  part  of  the  data  is  used.  The  Geometric 
De-Eutrophication  model  appears  to  be  slightly  better  than  the  Jel inski -Moranda 
De-Eutrophication  model  for  data. 


iV.V 


9.0  MODIFIED  GEOMETRIC  DE-EUTROPHICATION  MODEL 
9.1  Underlying  Assumptions 


a.  The  program  contains  an  unknown  number  of  errors. 

b.  Each  fault  in  the  program  is  independent  of  othe'-  fau'ts  an: 
each  of  them  is  equally  likely  to  cause  a  failure  during  tesfr 

c.  The  number  of  faults  detected  in  any  time  interval  is  incesence 
of  that  in  any  other  time  interval. 

d.  The  error  correction  time  is  negligible.  Eacn  e^or  csscve-ec 
is  immediately  removed. 

e.  No  new  errors  are  added  during  the  debugging  process. 

f.  The  program  is  not  being  altered  except  for  error  correct: on . 


9.2  Key  Features 


The  hazard  function  is  of  the  form 

2 ( t i )  =  DKMi ‘ 1 


with  D 
K 


Mi-1 

Mi 


the  fault  detection  rate; 

a  positive  constant  less  than  1; 

the  cumulative  number  of  errors  detected;  and 

the  cumulative  number  of  errors  found  up  to  the  i-th  time 
interval . 


The  reliability  function  is 


R(t)  *  €  -  DKH"t 


and  the  Mean  Time  to  Failure  is 


MTTF  = 


1 

DKM"  ’ 


with  n  =  the  total  number  of  time  intervals. 


9.3  Study  Results 


This  model  was  not  verified  with  any  test  data. 

10.0  SHOOMAN  EXPONENTIAL  MODEL 


10.1  Underlying  Assumptions 


a.  The  numoe-  of  errors  in  a  program  is  a  constant  and  decreases 
directly  as  errors  are  corrected. 

b.  The  er-'or  detection  rate  (failure  rate)  is  proportional  to 
the  numoer‘  of  remaining  errors. 

c.  The  total  number  of  machine  language  instructions  remains  constant 

o.  Operational  software  errors  occur  due  to  the  occasional  traversing 
of  a  portion  of  the  program  in  which  a  software  bug  is  hidden. 

e  Each  error  has  an  equal  chance  of  being  detected. 

f.  Software  errors  occur  with  a  probability  distribution  of 

f(t)  =  xexp  (-xt) 


where  t  =  CPU  operating  time;  and 

X  =  a  constant  of  the  hazard  function,  z(t). 


10.2  Key  Features 


*  is 


The  total  number  of  errors  remaining  in  the  program  debug  time 


er(t)  =  (E/I)-ec(r) 


where  E  =  the 
I  =  the 
ec(t)  =  the 


total  number  of  errors  present  at  time  t  =  0; 
total  number  of  machine  instructions;  and 
total  number  of  errors  corrected  in  interval  t. 


The  hazard  function  is  of  the  form 


z ( t )  =  Cer{  x) 

with  C  =  a  constant  of  proportional i ty. 

The  reliability  function  is 

R(t)  =  exp  {-C[(E/I )  -  ec(-r)]t} 
and  the  Mean  Time  to  Failure  is 

MTTF  =  l/{ C[(E/I )  -  ec(T  )]} . 


10.3  Study  Results 

The  Shooman  exponential  model  reduces  to  the  Jel inski-Moranda 
De-Eutrophication  model.  This  model  requires  a  sequence  of  times  between 
failures  in  order  to  estimate  the  parameters. 
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TECHNICAL  REPORT 


DEFINTION  OF  THE  FAULT  TOLERANT  SYSTEM 
RELIABILITY  MODEL  INTERFACES  WITH 
THE  SOFTWARE  RELIABILITY  MODEL 


1.0  INTRODUCTION 


1.1  Background 

Interface  checks  are  an  attractive  form  of  error  detection 
(at  least  as  far  as  a  program  running  on  the  interface  is  concerned) 
since  they  are  performed  automatically  and  efficiently  -  often  in  parallel 
with  the  execution  of  the  requested  operation  -  and  cannot  be  suppressed 
by  a  programmer.  However,  interface  checks  can  only  check  for  correctness 
of  use  of  an  interface  and  cannot  check  whether  any  usage  corresponds 
to  that  of  a  correct  program.  It  has  been  assumed  that  interface  exceptions 
and  failure  exceptions  indicated  the  presence  of  design  faults  in  the 
program  and  component  faults  in  the  interpreter,  respectively.  If  this 
was  always  the  case  then  the  implementation  of  fault  tolerance  by  the 
program  would  be  much  simplified  since  there  would  be  a  direct  relationship 
between  the  type  of  a  fault  and  a  particular  exception. 

1.2  Objectives  of  the  Research 

To  define  the  interface  of  the  software  reliability  model  with 
the  fault  tolerant  system  reliability  model,  it  is  necessary  to  look 
at  the  corresponding  outputs  and  required  inputs.  It  is  desired  that 
the  interface  definition  permit  stand  alone  use  of  the  software  reliability 
model  with  the  outputs  serving  as  inputs  to  the  fault  tolerant  system 
reliability  model  or  a  combined  operation  of  the  hardware  and  software 
models  of  the  state  space.  The  outputs  and  inputs  are  discussed  in  detail 
below. 


ye 


2.0  CHARACTERISTICS  OF  THE  SOFTWARE  RELIABILITY  MODEL 
AND  THE  FAULT  TOLERANT  SYSTEM  RELIABILITY  MODEL 


2.1  Software  Reliability  Model  Outputs 

Software  reliability  models  that  are  applicable  to  real-time 
software  address  both  the  dynamic  and  static  measurements.  The  corresponding 
outputs  are: 

(1)  Dynamic  measurements 

•  hazard  rate,  z(t) 

•  reliability  function,  R(t) 

•  mean  time  to  failure,  MTTF 

(2)  Static  measurements 

•  total  number  of  errors 

•  total  number  of  remaining  errors 

2.2  Fault  Tolerant  System  Reliability  Models 

In  accordance  with  the  "Automated  Reliability  and  Failure  Effects 
Method  for  Digital  Flight  Control  and  Avionic  Systems,  Volume  I:  Evaluation" 
this  report  will  focus  on  the  top  two  models.  The  evaluation  lists  the 
top  two  models,  in  order  of  preference,  as  CARSRA  (Computer  Aided  Redundant 
System  Reliability  Analysis)  and  CARE  II  (Computer  Aided  Reliability 
Estimation,  version  II).  A  different  evaluation,  sponsored  by  NASA  Langley 
Research  Center,  indicates  that  the  CARE  III  model  is  "best  suited  for 
evaluating  the  reliability  of  advanced  fault-tolerant  systems  for  commercial 
air  transport. "(1)  Therefore,  this  report  will  discuss  the  updated  and 
improved  CARE  model:  CARE  III.  In  addition,  N-Version  Software  (NVS), 
obtained  from  mul ti -version  programming,  will  be  considered  since  it 
is  a  primary  method  for  providing  software  fault  tolerance  and  was  not 
covered  in  the  above  reports. 


(1)  "Evaluation  of  Reliability  Modeling  Tools  for  Advanced  Fault-Tolerant 
Systems",  AIRLAB  INTERFACE,  NASA  Langley  Research  Center,  Hampton,  Virginia, 
December  1983,  p.  2. 


2.2.1  GARSRA  Inputs 
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The  following  items  comprise  the  required  input  data  and  the 
corresponding  variable  names  that  are  used  by  the  program. 

•  number  of  non-dependency  stages,  NIS 

•  number  of  dependency  stages,  NDS 

•  assigned  state  number,  NST 

•  dimension  of  the  stage,  NDIM 

•  number  of  modules  in  the  stage,  MOON 

•  transition  rates,  LMDA  (NST,  K,  J)  with  J=l,  2, ...NDIM  and 

K=1 ,  2. _ (NDIM-1) 

•  transitional  readiness  time  span,  AMT,  and  time  increment, 

ADT 

•  failure  probability  time  span,  FPMT,  and  time  increment, 

FPDT 

•  number  of  dependency  modules  in  the  system,  NARY 

•  each  dependency  module,  NIND  (I)  with  1  =  1,  2, ...NARY,  specified 
by  NIND  and  NDEP 

•  number  of  functional  readiness  configuration  entries,  NAV 

•  each  configuration  is  characterized  by  up  to  three  failed 
modules,  NA  (I,  K),  with  K=l,  2,  3  where  the  module  is  indicated 
by  XXY  with  XX  being  the  stage  number  and  Y  the  module  number 
within  the  stage 

•  number  of  stage  failure  patterns  equivalent  to  system  success, 
NOSCOF 

•  success  configurations,  ICQF  (I,  J)  with  I=N0SC0F  and  J=l, 

2, ...50 


•  accuracy  indicator,  NACCUR 


2.2.2  CARE  III  Inputs 


The  following  input  data  is  required  to  describe  the  system, 
he  corresoondi ng  variable  names  are  given  after  the  description. 


•  variable  which  defines  if  all  of  the  fault  handling  models 
have  exponential  distributions  only,  MARKOV 

•  numbe'-  of  fault  types  to  be  included  in  the  model,  NFTYPS 

•  parameter  for  transition  between  the  active  state  (A  or 
Ac)  to  the  detecteo  state  (D),  DEl(i) 

•  parameter  for  transition  from  the  active  state  A  to  the 
active  error  (erroneous  operation  state)  Ac,  RHO(i) 

•  parameter  for  transition  from  the  error  producing  state 

to  the  detected  state  or  to  a  single  fault  failure,  EPS(i) 

•  indicator  variable  defining  if  the  DEL  parameter  is  for 
an  exponential  or  uniform  density,  IDELF(i)  -  disregard 
if  it  is  a  Markovian  model 

•  indicator  variable  defining  if  the  RHO  parameter  is  for 
an  exponential  or  uniform  density,  IRHOF(i)  -  disregard 
if  it  is  a  Markovian  model 

•  indicator  variable  defining  if  the  EPS  parameter  is  for 
an  exponential  or  uniform  density,  IEPSF(i)  -  disregard 
if  it  is  a  Markovian  model 

t  probability  that  a  faulty  operation  will  be  successfully 
masked  by  the  system,  C(i) 

•  probability  that  a  module  detected  as  faulty  in  an  active 
state  A  is  identified  as  a  permanent  fault  and  isolated 
from  the  system,  PA ( i ) 

•  probability  that  a  module  detected  as  faulty  in  a  benign 
state  is  identified  as  a  permanent  fault  and  isolated  from 
the  system  (for  intermittent  or  transient  fault),  P6(i) 

•  exponential  rate  (intermittent  or  transient  fault)  for 
transition  from  an  active  state  (A  or  A^)  to  a  benign  state 
(B  or  Be),  ALP(i) 

•  exponential  rate  (intermittent  fault)  for  transition  from 
a  benign  state  (B  or  Bg)  to  an  active  state  (A  or  A^), 

BET ( i  ) 
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•  flag  for  outputting  the  moments  of  single  and  double  fault 
coverage  functions,  CVPRNT 

•  flag  for  a  plot  of  the  single  and  double  fault  coverage 
functions,  CVPLOT 

•  Y-axis  scale  for  plotting  coverage  functions,  IAXSCV 

•  parameter  governing  the  step  doubling  rule  used  in  the  solution 
DBLDF 

•  coverage  function's  truncation  value,  TRUNC 

•  number  of  stages  in  the  system,  NSTGES 

•  numbe”-  of  identical  modules  in  stage  number  x,  N(x) 

•  minimum  number  of  modules  needed  for  stage  ISTG  to  be 
operational  ,  M(x) 

•  operational  configurations  for  stage  x,  NOP  (i,x) 

•  option  for  the  output  printout,  IRLPCD 

•  option  for  the  summary  information  to  be  plotted  against 
time,  RLPLOT 

•  axes  specification  for  the  summary  information  plot,  IAXSRL 

•  number  of  fault  types  that  a  stage  is  subject  to,  NFCATS(x) 

•  fault  types  specification  for  stage  x,  JTYP(j,x) 

•  parameter  m  of  the  Weibull  fault  occurrence  rate  Xu)(xt)-°"1 
for  fault  type  j  for  stage  x,  0MG(j,  x) 

•  parameter  X  of  the  Weibull  fault  occurrence  rate  for  the 
fault  type  j  for  stage  x,  RlM(j,  x) 

•  flight  time  for  wnich  the  system  is  to  be  assessed,  FT 

•  time  scale  used  for  the  flight  time,  ITBASE 

•  number  of  equal  steps  that  the  flight  time  is  divided  into, 
NSTEPS 

•  flag  indicating  whether  or  not  the  system  fault  tree  is 
to  follow,  SYSFLG 

•  flag  indicating  whether  or  not  a  critical  pairs  fault  tree 
is  provided,  CPLFLG 
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•  parameter  used  to  limit  the  number  of  terms  used  in  computing 
the  coverage  failure  probability,  PSTRNC 

•  parameter  used  to  limit  the  number  of  fault  vectors  used 
in  computing  the  probability  of  system  failure  due  to  a 
lack  of  coverage,  QPTRNC 

•  parameter  affecting  the  computation  of  the  summary  information, 
KWT 

•  identification  label  for  the  system  represented,  TITLE 

•  logic  statements  to  form  a  single  system  fault  tree 

•  identification  label  for  the  critical  pair  tree,  TITLE 

•  logic  statements  for  the  critical  pair  tree 

2.2.3  NVS  Inputs 

The  following  parameters  are  necessary  for  the  NVS  reliability 

analysis. 

•  number  of  active  versions,  n 

t  error  probability  of  version  #i  (with  q-j  =  l-p-j),  p-j 

•  correlation  coefficient  between  #i  and  #j  (with  Pi ,  j  =  l“Pi ,  j ) , 

Pi ,  j 

•  maximum  number  of  versions  allowed  to  be  faulty  at  any  crosscheck 
point,  m* 

•  maximum  number  of  versions  allowed  to  be  faulty  in  common 
mode,  f* 


3.0  INTERFACE  DEFINITION 

It  is  desired  that  the  software  reliability  model  be  able  to 
be  used  in  a  stand  alone  environment  or  in  conjunction  with  the  fault 
tolerant  system  reliability  model.  Therefore,  the  interface  definition 
shall  dictate  which  outputs  of  the  software  reliability  model  must  serve 
as  inputs  to  the  fault  tolerant  system  reliability  model.  For  clarification, 
this  will  be  done  individually  for  the  three  models:  CARSRA,  CARE  III, 
and  NVS. 
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3.1  Interface  Definition  with  CARSRA 


The  required  inputs  for  the  CARSRA  model  are  from  three  different 
sources:  (1)  inputs  into  the  software  reliability  model;  (2)  outputs 
from  the  software  reliability  model;  and  (3)  inputs  only  for  the  fault 
tolerant  system  reliability  model.  Table  1  shows  the  distribution  of 
the  various  CARSRA  inputs  between  these  three  categories.  The  inputs 
from  the  first  two  categories  are  transferred  from  the  software  reliability 
model  into  the  fault  tolerant  system  reliability  model  at  the  interface. 

Hence,  the  interface  definition  for  the  CARSRA  model  is  shown  in 
Figure  1.  Table  2  shows  the  relationship  between  the  various  outputs 
from  the  software  reliability  model  and  the  CARSRA  inputs  (given  in  the 
second  column  in  Table  1). 

3.2  Interface  Definition  with  CARE  III 

Similar  to  the  CARSRA  model,  the  required  inputs  for  the  CARE 
III  model  are  from  the  following  three  sources:  (1)  inputs  into  the 
software  reliability  model;  (2)  outputs  from  the  software  reliability 
model;  and  (3)  inputs  only  for  the  fault  tolerant  system  reliability 
model.  The  distribution  of  the  CARE  III  inputs  amongst  these  three  categories 
is  shown  in  Table  3.  Figure  1  remains  applicable  for  defining  the  interface 
of  the  software  reliability  model  with  the  fault  tolerant  system  reliability 
model.  The  correspondence  between  the  software  rel iabi 1 i ty  model  output 
variables  and  the  CARE  III  input  variables  (given  in  the  second  column 
in  Table  3)  is  shown  in  Table  4. 

3.3  Interface  Definition  with  NVS 

For  the  NVS  model,  the  required  inputs  are  in  the  following 
two  categories:  (1)  outputs  from  the  software  reliability  model  and 
(2)  inputs  only  for  the  fault  tolerant  system  reliability  model. 

Table  5  shows  the  breakdown  of  the  NVS  inputs  into  these  categories  and 
Table  6  gives  the  relationship  of  the  NVS  inputs  to  the  software  reliability 
model  outputs. 
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4.0  CONCLUSION 

With  the  interface  definitions  as  described,  the  software  reliability 
model  is  able  to  be  used  in  a  stand  alone  environment  or  in  conjunction 
with  the  fault  tolerant  system  reliability  moael .  This  setup  is  useful 
for  error  detection.  The  first  stage  in  providing  fault  tolerance  is 
to  detect  errors  arising  from  the  execution  of  the  primary  module.  During 
its  execution,  the  module  will  be  subjected  to  the  interface  checks  provided 
by  the  underlying  system.  These  checks  could  detect  the  consequences 
of  faults  in  the  module  and  hence  signal  an  exception. 
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TECHNICAL  REPORT 


on 

FORMULATION  OF  THE  SOFTWARE  RELIABILITY  MODEL 

1.0  INTRODUCTION 

A  hierarchical  software  reliability  model  which  predicts  the 
reliaoility  of  software  prior  to  its  development  is  proposec.  This  moael 
snail  include  both  fault  tolerant  and  fault  intolerant  software  considerations 
With  this  model,  measurement  of  the  reliability  of  software  uncer  develop¬ 
ment  and  identification  of  the  data  to  be  collected  to  make  this  evaluation 
snal 1  be  possible. 


2.0  SOFTWARE  CHARACTERISTICS 


To  handle  both  fault  tolerant  and  fault  intolerant  software, 
the  rel iabi 1 i ty  model  shall  include  single  version  software,  N-version 
software,  decision  al gori thm( s ) ,  recovery  block(s),  and  acceptance  test(s). 
The  software  characteristics  of  each  of  these  design  techniques  are  dis¬ 
cussed  in  the  following  sections.  In  addition,  when  trying  to  specify 
software  reliability,  the  principal  concern  in  actually  to  describe  the 
ways  the  software  can  be  unreliable.  Software  reliability  may  be  charac¬ 
terized  by  a  profile  that  describes  the  modes  of  failure  that  the  software 
can  exhibit  as  a  consequence  of  faults  [DURHAM].  Therefore,  the  types 
of  faults  that  are  related  to  each  of  these  design  techniques  are  included 
in  the  following  sections. 


2.1  Single  Version  Software 


This  is  a  probabilistic  model  of  deterministic  or  random  events. 
Usually,  the  program  execution  is  deterministic,  while  the  development 


process  is  probabilistic.  Some  examples  of  faults  that  are  character'  sti c 
in  single  version  software  and  must  t.nerefore  be  accounted  for  are; 


a.  Incorrect  specification 

b.  Misunderstood  or  unclear  specification 

c.  Algorithmic  error  (sometimes  called  a  comoutati onal  or 
logic  error) 

d.  Input  data  error 

e.  Program  logic  error 

f.  Output  data  error 

2.2  N- Vers ion  Software 


N-Version  software  is  a  fault-tolerant  software  technique  wnich 
implements,  usually  in  parallel,  two  or  more  versions  that  are  functionally 
equivalent.  These  versions  may  be  produced  i ndependently  by  separate 
programming  teams  or  they  may  be  made  explicitly  different  through  examination 
and  subsequent  forcing  of  differences  into  the  versions  [KELLY].  Nevertheless 
when  the  alternate  versions  are  compared,  the  faults  should  be  distinguishable 
[HITT84J.  Some  of  the  faults  associated  with  N-version  software  include: 

a.  Specification  error 

b.  Performance  error  (due  to  incomplete,  inconsistent, 
or  ambiguous  specifications) 

c.  Non-termination  error 

d.  Algorithmic  error 

e.  Input  data  error 

f.  Output  data  error 

2.3  Decision  Algorithm 

The  decision  algorithm  determines  what  the  specific  output 
should  be.  The  decision  algorithm  may  be  a  majority  vote,  a  median  select, 
a  bit-by-bit  comparison  (with  the  number  of  bits  that  a**e  to  be  compared 
or  are  significant  specified),  or  an  average  [KELLY],  Some  considerations 
to  be  made  in  the  software  design  are: 

a.  The  type  of  decision  algorithm  used, 

b.  The  allowable  range  of  discrepancy  of  each  input  from 
all  other  inputs  to  the  decision  algorithm;  and 


c.  The  data  sensitivity  of  the  decision  algorithm. 


2.4  Recovery  Block 

The  recovery  block  method  is  a  faul t-tol erant  software  technique 
which  provides  alternate  components  which  may  be  switched  in  (usually 
serially)  to  take  the  place  of  a  faulty  component  that  has  been  rejected 
by  the  acceptance  test.  These  alternate  components  are  designee  independently 
from  the  main  software  component  (the  primary  alternate)  anc  generally 
only  provide  partial  functionality  of  the  software  component,  thus  reducing 
it  to  a  degraded,  simpler  mode.  Prior  to  entering  an  alternate,  the 
state  of  the  process  is  restored  to  that  current  just  before  entry  to 
the  primary  alternate  [RANDELL].  Some  examples  of  faults  that  occur 
in  the  software  for  recovery  blocks  include: 

a.  Specification  error 

b.  Performance  error 

c.  Non-termination  error 

d.  Algorithmic  error 

e.  Input  data  error 

f.  Output  data  error 

2.4.1  Forward  Recovery  Block 

A  forward  recovery  block  restores  the  system  to  a  consistent 
state  by  compensating  for  inconsistencies  found  in  the  current  state. 

For  a  single  process,  the  forward  recovery  block  technique  requires  a 
detailed  knowledge  of  the  extent  of  damage  done  and  a  strategy  for  fixing 
the  inconsistencies  [HITT36].  Therefore,  for  each  data  abstraction, 
exceptions  shall  be  specified  as  a  response  to  run-time  attempts  to  violate 
its  inherent  invariant  properties.  These  anticipated  faults  can  be  handled 
by  forward  recovery  block  techniques  [HITT84  and  CRIST  IAN ] . 

2.4.2  Backward  Recovery  Block 

Backward  recovery  block  techniques  involve  restoring  the  system 
to  some  previous  known  correct  state  (referred  to  as  rollback)  and  restarting 
the  computation  from  that  point  [HITT86J.  Unanticipated  faults,  i.e., 
design  faults,  can  be  handled  by  a  default  exception  handler  using  automatic 
backward  recovery  [HITT84  and  CRISTIAN]. 


2.5  Acceptance  Test 


An  acceptance  test  is  a  logical  expression  or  algorithm  which 
checks  the  acceptability  of  the  results  (or  input)  that  are  generated 
by  a  software  component  [RANDELL].  The  faults  that  are  associated  with 
an  acceptance  test  include: 

a.  Specification  error 

b.  Performance  error 

c.  Algorithmic  error 

d.  Input  data  error 

e.  Output  data  error 

2.6  Rollback 

The  rollback  recovers  the  input  state  of  the  software  to  its 
condition  prior  to  when  an  incorrect  or  faulty  version  was  run.  This 
resets  the  software  to  the  input  state  necessary  to  run  the  next  version. 

A  rollback  is  used  in  connection  with  a  recovery  block  and  hybrid  N-version 
software  systems.  Faults  that  are  characteri stic  of  rollback  are: 

a.  Specification  error 

b.  Input  data  error 

c.  Output  data  error 

d.  Unrecoverable  state 

2.7  Roll -Forward 

The  roll -forward  is  always  used  in  connection  with  a  forward 
recovery  block.  The  roll-forward  transfers  the  restored  state  obtained 
from  the  forward  recovery  block  to  a  forward  position  in  the  system. 

The  forward  position  for  this  transfer  depends  upon  the  state  for  which 
the  forward  recovery  block  has  compensated.  Faults  associated  with  roll- 
forward  include: 

a.  Specification  error 
b  Performance  error 

c.  Input  data  error 

d.  Output  data  error 


3.0  SOFTWARE  INTERFACES 


Software  reliability  is  a  probabilistic  measure  ana  is  defined 
as  the  probability  that  a  software  error  whicn  causes  discrepancies  from 
specified  requirements  in  a  specified  environment  aoes  not  lead  to  a 
failure  during  a  specified  exposure  period. 

3.1  Inputs 

The  inputs  to  this  software  reliability  model  are  the  incividual 
probabilistic  reliability  values  (or  safety,  availability,  or  accuracy 
values,  if  desired)  for  the  function  blocks.  These  values  are  either 
obtained  from  the  software  reliability  data  base,  estimated  by  the  lines 
of  code  (and  the  language),  derived  experimentally  by  subjecting  the 
function  block's  software  to  a  number  of  test  cases  and  counting  the 
failures  to  determine  a  reliability  value,  or  from  lower  (detailed)  level 
models.  The  inputs  should  be  real  numbers  with  a  range  of  0.0  < 
probabilistic  reliability  value  (or  safety,  availability,  or  accuracy 
value)  _<  1.0. 

3.2  Outputs 

The  output  of  the  software  reliability  model  is  the  overall 
probabilistic  reliability  value  (or  safety,  availability,  or  accuracy 
values,  if  desired)  of  the  closed  loop  block  diagram.  The  output  can 
also  be  for  different  levels  within  the  hierarchical  software  reliability 
model,  ranging  from  simple,  high  level  block  diagrams  to  complex,  detailed 
block  diagrams. 

3.3  Conmum'  cations 

When  first  developed,  a  function  block  may  be  considered  to 
be  highly  reliable,  but  if  the  software  of  that  function  is  subsequently 
rarely  used  or  tested,  the  confidence  in  that  reliability  value  may  be 


much  lower  than  it  would  be  with  extensive  use  and  testing.  Two  such 
examples  are  a  backup  bus  controller  and  the  auto  land  capability  on 
some  aircraft.  (The  auto  land  capability,  on  some  aircraft,  is  checked 
only  while  the  aircraft  prepares  for  takeoff  and  then  not  again  during 
the  entire  flight  until  it  is  actually  needec  for  landing.) 

4.0  SOFTWARE  FUNCTIONS 

The  model  is  represented  using  control  system  notation  for 
model  representation.  Each  software  module  is  considered  to  represent 
a  transformation  of  input  to  output.  While  a  signal  flow  graph  could 
be  used,  a  simulation  diagram  equivalent  to  the  flow  graph  has  been  selected 

4.1  Menu  Selection 

Each  module  can  be  represented  by  a  transfer  function  whose 
type  is  a  unique  icon.  The  software  shall  enable  menu  selection  of  the 
following  icons: 

a.  Structure  Icons 

•  single  version  software 

•  N-version  software  (the  number  of  versions  must  be 
specified  by  the  user) 

•  decision  algorithm 

•  recovery  block  (the  number  of  alternates  must  be 
specified  by  the  user) 

•  acceptance  test 

•  rollback 

•  roll-forward 

b.  Transfer  Icons 

•  forward  path 

t  positive  feedback 

•  negative  feedback 

‘  •  positive  feed-forward 

•  negative  feed-forward 


It  shall  be  possible  to  place  these  icons  along  a  display  such 
that  a  block  diagram  is  formed.  Each  of  the  structure  icons  shall  represent 
a  function  in  the  software  that  is  under  development. 


Figure  1.  General  Format  for  N-Version  Software 


4.1.1  Placement  Requirements 

The  structure  icons,  given  in  Section  4.1. a,  are  listed  as 
independent  entities.  However,  wnen  N-version  software  is  chosen,  it 
must  be  coupled  with  a  decision  algorithm  in  the  general  formats  depicted 
in  Figures  1-4. 

Figure  3  represents  an  N-version  software  model  in  which  only 
x  of  the  versions  are  run  at  a  time.  If  these  x  versions  fail  at  the 
decision  algorithm,  then  the  software  is  "rolled  back"  (or  restored  to 
the  original  input  state),  and  another  x  versions  are  run.  This  cycle 
will  continue  until  the  decision  algorithm  passes,  the  software  "times 
out"  (reaches  its  maximum  time  limit),  or  all  of  the  versions  have  been 
run  [SONERIU] . 
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A  variation  of  the  forward  recovery  block  format  might  be: 
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Figure  7.  Alternative  Format  for  a  Forward  Recovery  Block 

Section  4.1  lists  the  decision  algorithm  and  acceptance  test 
independently  of  the  N-version  software  and  recovery  blocks  to  allow 
for  the  variations  in  the  format.  This  permits  the  decision  algorithm 
and  acceptance  test  to  be  used  independently,  as  well  as  in  conjunction 
with  their  respective  pairs  [(N-version  software  and  decision  algorithm) 
and  (recovery  block  and  acceptance  test)].  Keeping  the  decision  algorithm 
and  acceptance  test  as  separate  entities  requires  that  each  N-version 
software,  decision  algorithm,  recovery  block,  and  acceptance  test  module 
have  individual  reliability  values.  This  should  improve  the  accuracy 
of  the  transfer  functions  for  this  portion  of  the  diagram  since  it  will 
accommodate  variations  in  the  implementation  of  these  concepts.  (Reference 
Section  6.0). 

4.2  Federal  Aviation  Administration  (FAA) 

Function  Criticality  Categories 

The  system  functions  shall  be  classified  as  critical,  essential, 
or  non-essential,  according  to  the  effects  of  malfunctions  or  design 
errors.  The  categories  are  defined  as: 
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Critical  -  Functions  for  which  the  occurrence  of  any 
failure  condition  or  design  error  would 
prevent  the  continued  safe  flight  and 
landing  of  the  aircraft. 

Essential  -  Functions  for  which  the  occurrence  of  any 
failure  condition  or  design  error  would 
reduce  the  capability  of  the  aircraft  or 
the  ability  of  the  crew  to  cope  with  adverse 
operating  conditions. 

Non-Essential  -  Functions  for  which  failures  or  design 
errors  could  not  significantly  degrade 
aircraft  capability  or  crew  ability. 


The  most  critical  function  of  a  system  will  determine  the  category  of 
the  whole  system  unless  that  system  has  been  partitioned  into  elements 
having  different  categories.  Correspondingly,  the  software  levels  used 
throughout  this  report  are  Level  1,  Level  2,  and  Level  3.  The  software 
level  required  for  certification  of  functions  is  based  upon  the  applicable 
criticality  category.  Level  1  is  associated  with  the  critical  category. 

Level  2  with  the  essential  category,  and  Level  3  with  the  non-essential 
category  [RTCA]. 

4.3  Function  Block  Reliability 

It  shall  be  possible  to  determine  the  transfer  function  ("reliability") 
for  the  function  blocks  in  the  block  diagram  through  the  use  of  a  software 
reliability  model  which  addresses  dynamic  measurements.  These  transfer 
functions  are  often  available  through  previous  research  and  will  consequently 
be  furnished  in  the  software  reliability  data  base.  (Reference  Subtask 
4.4.4,  Oefine  Data  Required  for  the  Software  Reliability  Data  Base  and 
Set  Up  the  Data  Base). 

Furthermore,  the  FAA  criticality  categories  will  be  supplied 
for  each  of  the  transfer  functions  to  identify  which  of  the  function 
blocks  or  parameters  strongly  effect  the  overall  system  criticality. 

Any  variation  in  the  criticality  of  these  function  blocks  would  have 
a  dramatic  effect  on  the  overall  criticality  estimate  and  its  associated 
confidence  level . 
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4.3.1  Detailed  Function  Blocks 


This  hierarchical  software  reliability  model  may  be  used  with 
varying  levels  of  detail  [and  consequently  will  provide  varying  degrees 
of  accuracy  (Reference  Section  6.1.)].  Figure  8  gives  a  simple  example 
of  a  possible  situation.  In  this  example,  reliability  values  (probability 
of  the  software  functioning  correctly)  may  be  substituted  for  each  function 
block.  This  will  pennit  the  determination  of  the  overall  system  reliability. 
(Reference  Section  4.5.)  However,  this  is  a  very  high  level  model,  and 
as  such,  the  accuracy  of  the  reliability  values  tend  to  be  not  as  good 
as  might  be  desired.  Each  of  the  software  design  techniques  (single 
version  software,  N-version  software,  decision  algorithm,  recovery  block, 
and  acceptance  test)  can  actually  be  broken  down  into  more  detailed  function 
blocks,  dependent  upon  the  possible  faults  associated  with  the  software. 

These  faults  were  discussed  in  Sections  2.1  through  2.7. 

A  detailed  diagram  for  the  single  version  software  function 
block  and  the  decision  algorithm  function  block  are  given  in  Figure  9. 

With  reference  to  Sections  2.1  and  2.3,  Tables  1  and  2,  respectively, 

show  the  association  between  the  identified  faults  and  the  portion  of 

the  diagram  in  Figure  2  in  which  they  would  occur.  Therefore,  the  reliability 

of  each  of  the  detailed  function  blocks  in  Figure  8  is  a  probability 

of  success  for  that  portion  [or  1.0  -  (  the  probability  that  the  associated 

fault(s)  listed  in  Table  1  or  2  for  the  detailed  function  block  will 

occur)] . 

4.3.2  Function  Block  States 

The  four  possible  states  for  any  of  the  function  blocks  (detailed 
or  not)  are: 

a.  the  function  block  fails  (an  error  is  detected  in  the 
function  block)  and  it  is  corrupt  (contains  one  or  more 
error) ; 

b.  the  function  block  passes,  yet  it  is  corrupt; 

c.  the  function  block  fails  and  it  is  error  free;  and 

d.  the  function  block  passes  and  it  is  error  free. 


Detailed  Function  Blocks 


Paul ts 


:npuL  i n  put 

Validity  Integrity 


ilqorithm  Fo>"nat 


Input/ 
Outout 
Integri ty 


incorrect 
Speci f i cation 


Misunderstood 
or  Unclear 
Soecification 


A1 gori thmi c 
Error 


Input  Data 
Error 


Program 
Logic  Error 


Output  Data 
Error 


Table  1.  Correlation  of  Faults  with  the  Detailed  Portions 
of  the  Single  Version  Software  Function  Block 
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4.4  Software  Reliability  Data  Base 

A  software  reliability  data  base  shall  be  established  to  store 
reliability  values  for  the  various  function  blocks  identified  in  the 
software  reliability  model.  These  software  reliability  values  will  be 
collected  from  research  performed  and  documented  in  technical  reports. 

The  use  of  these  reliability  values  will  provide  a  more  accurate  estimation 
of  the  software  reliability  and  the  confidence  associated  with  this  estimate. 

4.5  System  Reliability 


The  function  blocks  will  each  have  an  associated  transfer  function 
( "rel iabi 1 i ty" ) .  The  overall  system  reliability  is  determined  via  block 
diagram  reduction  techniques,  thus  giving  the  overall  system  transfer 
function . 

In  this  software  reliability  model,  the  signal-flow  diagram 
reduction  technique  is  used  to  determine  the  overall  system  transfer 
function  ("reliability").  The  si gnal -fl ow  diagram  is  useful  in  analyzing 
multiple-loop  feedback  systems  and  in  determining  the  effect  of  a  particular 
element  or  parameter  in  an  overall  feedback  system,  whereas  the  block 
diagram  is  useful  in  the  design  and  analysis  of  sections  of  a  feedback 
system.  Block  diagram  reduction  techniques  become  tedious  and  time  consuming 
as  the  number  of  feedback  paths  increases.  To  solve  complex  problems, 
it  is  much  simpler  to  use  the  theorems  and  properties  of  signal-flow 
graphs . 

The  equations  used  in  this  analysis  follow  S.  J.  Mason's  theorems 
on  the  properties  of  signal-flow  graphs.  The  general  expression  for 
the  (closed  loop)  system  transfer  function  using  the  signal-flow  diagram 
reduction  technique  is  given  by 

r  r.  » 

KbK  K 


Reliability  = 


4  =  1  -  £Li  +  EL2  -  "L3  +  ...  +  (-l)n  ~Ln , 

Ll  =  the  gain  of  each  closed  loop  in  the  graph, 

l_2  =  the  product  of  the  loop  gains  of  any  two  non-touching 
closed  loops, 

l_3  =  the  product  of  the  loop  gains  of  any  three  non-touching 
closed  loops, 

Ln  =  the  product  of  the  loop  gains  of  any  n  non-touching 
closed  loops, 

G«  =  the  gain  of  the  Kth  forward  path, 

di(  =  the  value  of  t  for  that  part  of  the  graph  not  touching  the 
Kth  forward  path  [SHINNESS]. 

The  transfer  function  for  N-version  software  and  recovery  blocks 
are  dependent  upon  the  number  of  versions  or  alternates  (n).  For  N-version 
software,  the  transfer  function  is 

Cn 

1  -  n  (1  -  «i) 

i  =  1 

wi  th 

n  =  the  number  of  versions  in  the  N-version  software; 

C(n,r)  =  the  number  of  r  combinations  of  an  n  element  set; 

Cn  =  C ( n  ,z  ) ; 

«  =  the  product  of  reliabilities  of  the  i-th  combination 
required  for  success; 

i  =  1,  2,  3,  . . .  .Cn; 

z  =  [(n/2)  +  1]  if  n  is  an  even  number;  and 
z  =  [(n  +  l)/2]  if  n  is  an  odd  number. 

For  a  recovery  block,  the  transfer  function  is 

Gi  +  (1  -  Gi)G2  +  (1  -  Gi)(l  -  G2)G3  + 

with  Gi  =  the  reliability  value  for  alternate  i  and 
i  -  1,  2,  3,...n. 


The  reliability  values  (transfer  functions)  for  the  hybrid 
N-version  software  and  the  recovery  "block  will  vary  if  not  all  of  the 
n  versions  or  n  alternates  are  used.  The  above  equations  will  give  a 
higher  reliability  value  than  the  actual  situation  in  these  cases.  Sections 
5.1.1  and  6.1.2  discuss  the  accuracy  of  the  reliability  values  for  the 
N-version  software  and  recovery  block  and  how  they  can  be  calculated 
to  reflect  the  actual  situation.  (See  Appendices  I  and  II  for  some  examples 
with  these  equations). 

Although  the  transfer  function  for  most  of  the  function  blocks  is 
the  reliability  value,  one  exception  to  this  is  with  rollback  or  any 
feecback  block.  For  any  feeGback  path,  the  transfer  function  of  the 
equivalent  block  in  the  path  is  (1.0  -  reliability  value).  (See  Appendix 
III  for  a  further  explanation  and  proof).  The  second  exception  is  with 
feed-forwarc  paths.  The  transfer  function  of  the  equivalent  block  in 
a  feed-forward  path  (for  example,  rol 1-forward )  is  (1.0  -  reliability 
value).  (Refer  to  Appendix  IV  for  additional  information.) 

4.5.1  Simple,  High  Level  Model  Example 

For  a  sample  problem  involving  a  simple,  high  level  model, 
the  example  given  in  Figure  8  will  be  used.  The  Single  Version  Software 
will  be  a  commonly  used  algorithm  or  process,  and  therefore,  it  will 
have  a  high  reliability  which  is  well  documented  and  stored  in  the  software 
reliability  aata  base.  For  this  example,  the  reliability  will  be  0.9991. 

The  N-Version  Software  will  have  three  indepedent  versions, 
running  in  parallel.  This  is  a  common  form  of  N-Version  Software,  but 
not  one  with  the  highest  reliability.  Through  the  user's  tests,  it  is 
determined  that  this  function  block  will  have  a  reliability  of  C.994. 

The  Decision  Algorithm  will  take  an  average  of  the  outputs 
from  the  N-Version  Software,  excluding  any  version  which  does  not  meet 
the  timing  constraints.  This  type  of  Decision  Algorithm  has  a  high  reli¬ 
ability  since  it  does  not  have  a  range  check  or  any  other  source  for 
determining  the  validity  of  the  output.  This  Decision  Algorithm  will 
not  eliminate  any  erroneous  outputs  and  will  not  detect  the  occurrence 


of  correlated  faults.  The  decision  algorithm  is  simple  (and  consequently 
highly  reliable),  but  it  is  not  always  the  most  desirable  since  its  simpli¬ 
city  detracts  from  its  capability  of  detecting  errors.  For  this  example, 
it  is  assumed  that  the  reliability  of  the  Decision  Algorithm  is  0.988. 

The  Recovery  Block  is  a  backward  recovery  block  with  a  primary 
alternate  ana  two  additional,  extremely  simplified  alternates  The  Recovery 
Block  is  of  a  common  form  and  its  reliability  can  be  obtained  from  the 
software  reliability  data  base.  For  this  example,  it  will  have  a  reliability 
value  of  0.97. 

The  Acceptance  Test  is  an  output  format  check.  This  is  a  simplistic 
algorithm  wnicn  is  commonly  used  in  various  models.  The  reliability, 
as  determined  from  the  software  reliability  data  base,  will  be  0.9997. 

Finally,  the  Rollback  recovers  the  input  state  of  the  software 
to  its  condition  upon  entry  to  the  Recovery  Block.  This  is  a  retrieval 
of  the  data  from  its  memory  location.  The  reliability  of  this  common 
form  of  Rollback  will  be  assumed  to  be  available  in  the  software  reliability 
data  base.  For  this  example,  the  reliability  is  0.9999.  This  makes 
the  transfer  function  for  the  Rollback  equal  to  (1.0  -  0.9999).  (Reference 
the  final  paragraph  in  Section  4.5.) 

Hence,  for  this  example, 

Li  =  (0.97)  x  (0.9997)  x  (+1.0  -  0.9999)  =  0.0000969709 

L?  through  Ln  =  0 

G],  =  (0.9991)  x  (0.994)  x  (0.988)  x  (0.97)  x  (0.9997) 

=  0.951467 

G2  through  G((  =  0 

4!  =  1 

4=1-  (+0.0000969709)  =  0.99990302 

Therefore , 


Reliability 


(0.951467)  x  (1) 
(0.99990302) 


0.951559 


Rel iabi 1 i ty  =  0.95. 
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4.5.2  Complex,  High  Level  Model  Example 

The  complex,  high  level  model  examole  will  be  as  shown  in  Figure 
10.  Block  (1)  is  a  Single  Version  Software  block.  For  this  example, 
it  will  be  a  simple  algoritnm  witn  a  '■eliability  value  of  0.998. 

Block  (2)  is  N-Version  Software  with  nine  independent  versions 
in  which  only  three  versions  are  run  at  a  time.  (This  type  of  N-Version 
Software  was  shown  in  Figure  3).  For  this  example,  it  is  assumed  that 
the  reliability  value  for  the  N-Version  Software  is  0.999. 

Block  (3)  is  the  Decision  Algorithm  for  the  N-Version  Software. 

In  this  examole,  the  Decision  Algorithm  will  be  a  median  select  with 
a  re  1 i a d i T i ty  value  of  0.983. 

Block  (4)  is  an  Acceptance  Test  which  will  check  that  the  range 
of  the  output  from  the  Decision  Algorithm  is  correct.  If  the  Decision 
Algorithm  failed,  then  the  software  will  Rollback  after  the  Acceptance 
Test.  Similarly,  if  the  Acceptance  Test  fails,  then  the  software  will 
Rollback  to  the  N-Version  Software.  Tne  input  to  the  Acceptance  Test 
will  be  stored  to  accommodate  for  the  Rollback  from  the  Recovery  Block. 

For  this  example,  the  reliability  value  for  the  Acceptance  Test  will 
be  0.992. 

Block  (5)  is  a  Recovery  Block  of  the  common  form  with  a  primary 
alternate  and  two  additional  alternates.  For  this  example,  the  reliability 
value  of  the  Recovery  Block  will  be  0.976. 

Block  (6)  is  the  Acceptance  Test  for  the  Recovery  Block.  In 
this  example,  it  is  assumed  that  the  reliability  value  for  this  Acceptance 
Test  is  0.995. 

The  Rollback  for  the  Backward  Recovery  Block  is  given  in  block 
(7).  This  is  a  retrieval  of  the  data  that  was  stored  prior  to  entry 
into  Acceptance  Test  #1.  For  this  example,  the  reliability  value  for 
this  Rollback  is  0.996.  Thus,  the  transfer  function  for  this  block  is 
(1.0  -  0.996). 

The  Rollback  for  the  N-Version  Software  is  given  in  block  (8). 

For  this  example,  the  reliability  value  is  0.997.  Consequently,  the 
transfer  function  is  (1.0  -  0.997). 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 


Single  Version  Software 
N-Version  Software 
Decision  Algorithm 
Acceptance  Test  #] 

Recovery  Block 
Acceptance  Test  #2 

Rollback  #1  --  Backward  Recovery  Block 
Rollback  #2  --  N-Version  Software  in  Which 
Only  x  Versions  are  Used  at  a  Time 
Acceptance  Test  #3 


Figure  10.  Complex,  High  Level  Model  Example 


Finally,  block  (9)  is  an  Acceptance  Test  which  checks  the  final 
output  against  the  input.  For  this  example.  Acceptance  Test  #3  will 
have  a  reliability  value  of  0.95,  giving  a  transfer  function  of  (1.0 
-  0.95). 

Hence,  for  this  example, 

Closed  Loop  1  =  (0.999)  x  (0.983)  x  (0.992)  x  (1.0  -  0.997) 

=  0.0029225 

Closed  Loop  2  =  (0.992)  x  (0.976)  x  (0.995)  x  (1.0  -  0.996) 

=  0.0038534 

Closed  Loop  3  =  (0.998)  x  (0.999)  x  (0.983)  x  (0.992)  x 
(0.976)  x  (0.995)  x  (1.0  -  0.95) 

=  0.0472068 

TLj  =  (Closed  Loop  1)  +  (Closed  Loop  2)  + 

(Closed  Loop  3) 

=  0.0029225  +  0.0038534  +  0.0472068 
=  0.0539827 
L2  through  Ln  =  0 

Gj  =  (0.998)  x  (0.999)  x  (0.983)  x  (0.992)  x  (0.976)  x  (0.995) 

=  0.944135 
G2  through  G«  =  0 

Ai  =  1 

4=1-  (0.053982?)  =  C. 9460173 

Therefore , 


Reliability  »  -I?-  944135 )  A ii )  =  0.9980103 

(0.9460173) 


Reliability  =  0.998. 
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4.5.3  Simple,  Detailed  Level  Model  Example 

An  example  of  a  simple,  detailed  level  model  is  given  in 
Figure  9.  For  this  example,  the  structure  icons  would  all  be  single 
version  software  and  the  transfer  icons  would  be  either  forward  path 
or  positive  feedback  (Reference  Section  4.1.).  Hence,  an  equivalent 
block  diagram  for  this  detailed  block  diagram  is  given  in  Figure  11. 

In  Figure  11,  block  (1)  is  a  Single  Version  Software  block 
which  checks  the  input  set.  This  is  a  common  process,  so  the  reliability 
value  will  be  assumed  to  be  available  in  the  software  reliability  data 
base.  For  this  example,  the  reliability  value  for  block  (1)  is  assumed 
to  be  0.98. 

Block  (2),  a  Single  Version  Software  block,  will  represent 
an  inDut  integrity  check.  It  will  be  assumed  that  this  algorithm  is  common 
and  can  consequently  be  found  in  the  software  reliability  data  base. 

For  this  example,  the  reliability  value  will  be  0.97. 

Block  (3)  is  a  Single  Version  Software  block  which  performs 
an  algorithm.  For  this  example,  the  algorithm  will  be  a  simple  one. 

Therefore,  the  reliability  value  for  this  transfer  block  will  be  assumed 
to  be  0. 992. 

Block  (4)  will  represent  an  output  format  check,  performed 
by  a  Single  Version  Software  block.  For  this  example,  the  reliability 
value  for  this  Single  Version  Software  will  be  0.996. 

The  final  Single  Version  Software  block,  block  (5),  will  be 
used  to  perform  an  input/output  integrity  check  in  which  the  output  is  checked  against 
the  input  to  verify  the  integrity  of  the  input  data.  For  this  examole, 
the  reliability  value  for  block  (5)  will  be  0.964.  Hence,  the  transfer 
function  for  this  block. will  be  (  1  0  -  0.964). 

Thus,  for  this  example, 

L]  =  (0.98)  x  (0.97)  x  (0.992)  x  (0.996)  x  (1.0  -  0.964) 

=  0.033812 
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Figure  11.  Equivalent  Diagram  Indicating  the  Structure  Icons 
to  be  Used  in  the  Detailed  Diagram  for  a  Single 
Version  Software  Function  Block 


l_2  through  Ln  =  0 

Gj  =  (0.98)  x  (0.97)  x  (0.992)  x  (0.996) 

=  0.9392232 
G2  through  G«  =  0 
A1  =  1 

t  =  1  -  (0.033812)  =  0.966188 

Therefore , 

Re  1  i  abi  1  i  ty 
Rel iabi 1 i ty 

4.6  Safety 

Safety  is  concerned  with  the  state  in  which  the  function  block 
fails  (an  error  is  detected),  but  the  function  block  is  error  free. 
Although  a  function  block  is  considered  to  be  unsafe  when  the  system 
is  unreliable,  safety  also  covers  this  extra  state.  Hence, 

safety  =  [(the  probability  that  an  error  exists  and  it  is 

detected)  +  (the  probability  that  no  error  exists)] 

or 

safety  =  [1  -  (the  probability  that  an  error  exists  and  it 
is  not  detected)] 

while  reliability  =  [(the  probability  that  an  error  exists 

and  it  is  detected)  +  (tie  probability 
that  no  error  exists  anc  no  e^or  is 
detected ) ] 

or 

reliability  -  |  1  -  [(the  probability  that  an  error  exists 

but  the  error  is  not  detected)  +  (the  proba 
bility  that  no  error  exists,  but  an  error 
is  detected)]  ; 


-  J0J392232)  x  _(  1)  =  0-972C916 

(0.966188) 

=  0.97. 


Therefore,  safety  >  reliability. 


Actually,  tne  software  reliability  mocel  can  be  used  to  determine 
the  safety  of  the  high  level  models.  Instead  of  using  reliability  values 
for  each  function  block,  in  the  determination  of  the  overall  transfer 
function,  if  the  safety  value  for  each  of  tne  function  blocks  is  used 
in  the  calculations,  tne  resultant  value  will  be  the  safety  of  the  overall 
system. 

4.7  Availability 

As  wit.n  reliability  ana  safety,  availability  can  be  determined 
through  tne  use  of  the  software  reliability  model.  To  do  so,  tne  availabil 
values  snoulc  be  usee  for  eacn  function  bloc<  in  tne  determination  of 
the  overall  transfer  function.  By  using  availability  values  instead 
of  reliability  values,  the  resultant  value  will  be  the  availability  of 
the  overall  system. 

The  availability  values  are  determined  as  follows: 

availability  =  [(the  probability  that  an  error  exists,  it  is 
detected,  and  it  is  corrected)  +  (the  proba¬ 
bility  that  no  error  exists  and  no  error  is 
detected )] 

or 

availability  r  (1  -  [(the  probability  that  an  error  exists, 

it  is  detected,  but  it  is  not  corrected)  +  (the 
probability  that  an  error  exists  and  no  error 
is  detected)  +  (the  probability  that  no  error 
exists  and  an  error  is  detected)]} 

Therefore,  0  £  availability  £  1.0.  By  comparison  to  reliability 
and  safety,  availability  £  reliability  £  safety. 


5.0  TIMING  CONSTRAINTS 


If  a  software  component  should  execute  in  1  msec.,  a  time'" 
could  detect  software  faults  tnat  cause  the  execution  time  to  exceed 
1  msec.  Many  of  the  reliability  mocel's  timing  constraints  deal  with 
the  fault  tolerant  portions  wne^e  the  entire  process  (fault  detection, 
damage  assessment,  recovery,  ano  fault  treatment)  must  take  place  fast 
enough  to  satis*./  real-time  requi  rements .  "No  matter  which  fault  tolerant 
software  method  is  usee,  real-time  systems  must  arrive  at  a  consistently 
correct  solution  within  the  time  frame  determined  by  the  control  system 
dynamics.  Failure  car,  occur  cue  to  excessively  long  response  times, 
e.g.,  tne  system  goes  unstaole  since  the  hard  ceadlines  for  code  execution 
are  mi ssed. " [HITT66] 


6.0  ACCURACY  CONSTRAINTS 


The  accuracy  and  reliability  of  tne  N-version  software,  decision 
algorithm,  recovery  block,  and  acceptance  test  are  deoendent  upon  the 
way  in  which  these  concents  are  implemented.  For  example,  N-version 
software  may  be  implemented  as: 


a.  two  independent  versions 

b.  three  indpendent  versions 

c.  more  than  three  independent  versions 

d.  an  N-version  software  model  in  which  only  x  versions 
are  run  at  a  time,  and  if  these  versions  fail  for  some 
reason,  then  x  or  less  of  the  remaining  (N-x)  versions 
are  run.  (This  is  deoictec  in  Figure  3  ) 

e.  an  N-Version  moce1  in  which  a  combination  of  x  versions 
are  run,  and  if  these  x  versions  fa:l  for  some  reason, 

a  different  comDination  of  x  versions  is  run,  and  so  on, 
(NOTE.  This  concept  is  different  from  item  d,  above, 
because  this  implementation  groups  different  combinations 
of  x  versions  item  d,  however,  uses  x  versions,  and  if 
they  fai’,  tne  x  versions  are  in  essence  thrown  out  and 
a  complete^  new  group  of  x  versions  are  used;  not  a  new 
combination  of  x  versions,  but  x  complete1y  new  versions.) 


Some  of  the  differences  which  affect  the  reliability  and  accuracy 
of  the  decision  algorithm  are: 


.'V.^>.r'<.wV.,T,V»V.T^V 
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a.  majority  voce; 

b.  mecian  select; 

c.  average;  anc 

d.  tne  oecision  algoricnm  only  considers  those  values  which 
are  in  certain  range  and  then  ft  uses  one  of  the  met. hoes 
(a ,  b ,  or  c) ,  above. 


The  recovery  b 1 o c < 1 s  accuracy  ana  reliability  depenc  on  the 
following  items  (to  name  a  few): 


a.  backward 

i.)  how  far  it  rolls  back;  and 
li.)  tne  numoer  of  alternates  available. 

b.  forward 

i.)  how  far  it  rolls  forward;  ana 
if.)  tne  accuracy  of  the  value(s)  assigned  prior  to 
the  roll. 


Some  of  the  concepts  that  affect  the  accuracy  and  reliability 
of  the  acceptance  test  depend  on: 


a.  the  range  of  the  values  accepted; 

b.  tne  rate  of  change  determination  for  the  variables;  and 

c.  the  format  of  the  data. 


Needless  to  say,  all  of  these  implementation  character!'  sti  cs 
must  be  considered  and  will  affect  the  accuracy  of  the  reliability  values. 
Sections  4.1  and  4.1.1  discuss  how  the  software  reliability  model  is 
set  up  to  accommodate  tne  imolementation  variations.  With  this  software 
reliability  model  design,  the  accuracy  is  improved  since  these  considerations 
can  be  taKen  into  account  in  the  assignment  of  reliability  values  (or 
accuracy  --  see  Sections  6.1  and  6.1.1  through  6.1.3  --  or  safety  or 
availability  values)  to  the  function  blocks. 


6.1  Accuracy 

The  accuracy  of  the  software  reliability  mooel  will  depend 
upon  the  accuracy  of  the  individual  values  that  are  used  as  the  transfer 
functions  for  eacn  of  the  function  blocks.  In  most  cases,  the  accuracy 
of  a  model  with  cetailed  function  blocks  will  be  bette1"  than  tne  high 


level  software  reliability  model  since  the  accuracy  of  the  individual 
transfer  functions  will  be  imorovec  (Re'e^ence  Section  4.3.1.}. 

The  accuracy  of  the  reliability  values  that  are  usee  for  the 
function  blocks'  transfer  functions  will  depena  upon  the  method  used 
to  obtain  such  values.  If  the  values  are  obtained  from  the  software 
reliability  data  base,  then  the  accuracy  of  these  values  are  indicated 
in  the  technical  reoorts  for  the  research  method  that  determined  the 
values.  If  the  values  a^e  obtained  through  the  use  of  a  different  software 
reliability  model,  then  the  accuracy  is  dependent  on  the  tyoe  of  mocel 
used.  Regardless,  accuracy  values  can  be  obtained  for  any  anc  all 
of  tne  transfer  functions,  anc  therefore,  an  accuracy  for  t.ne  overall 
software  reliability  value  can  be  calculated. 

6.1.1  Accuracy  of  the  Hybrid  N-Version  Software 

The  accuracy  of  the  hybrid  N-version  software  reliability  value 
(or  safety  or  availability  values?  depends  upon  the  numDer  of  versions 
that  are  actually  used.  (Reference  Figure  3  for  an  example  of  a  hybrid 
N-version  software.)  Usually,  if  the  software  is  run  and  only  y  of  the 
n  ve-sions  are  used,  then  the  reliability  value  (or  safety  or  availability 
values)  has  been  overrated  by  considering  the  additional  (n-y)  versions 
in  the  calculation  of  the  transfe1"  function  for  the  hybrid  N-version 
sc<twar"e.  Certainly,  if  it  is  known  that  only  y  of  the  n  versions  are 
actually  being  usee,  then  the  ca’culation  of  the  transfer  function  for 
tne  N-version  software  should  only  include  those  y  versions.  (See  Appencix 
I,  Example  3,  *or  a  demonstration  of  this  accuracy  effect.) 

6.1.2  Accuracy  of  the  Recovery  Block 


The  recovery  block,  by  definition,  consists  of  n  alternates 
Of  these  n  alternates,  only  one  is  run  at  a  time,  and  only  if  that  alternate 
fa-’s  will  the  software  rollback  and  run  the  next  alternate.  Hence, 
if  'ess  than  tne  n  alternates  are  actually  used,  the  reliability  (or 
safety  or  a va i 1  a b i 1 1 ty )  of  the  recovery  block  will  generally  decrease. 
Furthermore,  this  decrease  in  the  recovery  bio  s  ansfer  function 
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(reliability,  safety,  or  availability  value)  will  cause  a  decrease  in 
tne  overall  software  reliability  value  (or  safety  or  avai laoi 1  * ty  value), 
calculated  witn  the  software  reliability  mocel .  (See  Aooerc'x  ”  'O'* 
sene  recovery  block  calculations  that  address  the  effect  on  accuracy 
wren  fewe”  tnan  tne  n  alternates  are  actually  usee.) 

6.1.3  Accuracy  Example  for  the  Software  Reliability  Model 

To  demonstrate  the  use  of  the  software  reliability  ~cce'  for 
an  accuracy  calculation,  tne  simole  block  d"!  a  gran  snown  in  --cure  3  anc 
tne  example  in  Section  4.5.1  will  be  used.  Tne  first  seen  in  one's  deter¬ 
mination  is  to  assign  accuracy  values  to  each  of  tne  blocks .  For  ones 
examcle,  tne  following  values  will  be  used: 


Function  Block 

Accuracy 

Value 

Transfer 

Function 

Single  Version  Software 

+  0.0002 

0.9998 

N-Version  Software 

+  0.001 

0.999 

Decision  Algorithm 

+  0.003 

0.997 

Recovery  31ocx 

+  0.0C5 

0.995 

Acceotance  Test 

+  0.0001 

0.9999  j 

Rollback 

1 

+  0.00C05 

0.00CC5 

_ 

Table  4.  Accuracy  Values  for  the  Simple  Block  Diagram  Example 


'"ese  accuracy  values  reflect  the  accuracy  of  the  re! iabil i ty  val.es 
fat  are  used  in  the  simple,  high  level  mode!  examole. 

Cccosite  of  the  software  reliability  model  calculations,  tne 
f 3nsfer  functions  for  the  blocks  are  (10  -  .  accuracy  value  ',  except 
fbr  r^i ipack,  rol 1- forward,  tne  equivalent  block  witnm  a  feecbac-  Iood, 
on  tne  equivalent  block  within  a  feed-forwarc  patn,  wni :n  use  tne  absolute 
value  of  tne  accuracy  value.  Tne  respective  transfer  functions  are  listed 


’ n  Table  4 . 


Hence,  for  this  example, 


L;  =  (0.995)  x  (0.9999)  x  (0.00005)  =  0.0000-197 
L?  through  Ln  =  0 

Gi  =  ( 0 . 9998 >  x  (0.999)  x  (0.997)  x  (0.995)  x  (0.9999) 
*  =  (0.9907257) 

G£  tnrougn  Gi<  =  0 

-i  =  1 

1  =  1  -  0 . C0CC-97  =  0.9999502 


. ?e  rer  j  re  , 

• _  .  ,  !  (0.9907257  )  x  (  1)  ! 

“  1  “  I  - - - - - | 

(0.9999503)  | 

-  1  -  | 0 . 99077  49  j  *  +  0.0092251 

Accuracy  =  +  0. 00923. 

7.0  RESPONSE  TO  UNDESIRED  EVENTS 

The  software  reliability  model  described  herein  assumes  independence 
between  the  function  blocks  This  model  neglects  the  existence  of: 

a.  multiple  faults  wmch  produce  dissimilar  outputs  but  are 
manifested  by  the  same  input  conditions,  or 

b.  related  software  design  faults  causing  identical  incorrect 
outputs . 

Tne  e^o'-s  that  are  manifested  by  these  faults  are  known  as  coincident 
errors  and  cause  a  degradation  in  the  reliability  (or  safety  or  availability). 
Thprp^prp,  to  improve  tne  accuracy  of  the  software  reliability  model, 
the  urnc 1  dent  errors  must  be  considered.  This  might  be  done  with  an 
anai/s's  similar  to  that  suggested  by  Dave  5.  Ecknardt,  Jr  and  Larry 
D.  Lee.  The  analysis  makes  the  assumpt'ons  that  il)  tne  input  series 
X \ ,  X 2 , . . . . ,  's  stationary  and  independent  anc  (2)  the  versions  of  software 
components  designed  independently  [ECKHARDT'. 


When  evaluating  tne  peccability  of  coincident  errors,  the  area 
of  concern  is  the  N-version  software.  Inis  analysis  is  interested  in 
the  probability  that  z  or  more  of  tne  functions  fail  at  the  same  time, 
with  z  =  (n/2)  if  n  is  even  anc  z  =  [ ( r.  t  1 ) / 2 ]  if  n  is  odd.  The  following 
analysis  will  give  a  conservative  estimate  (maximum  possible)  of  the 
probability  of  coincident  errors  for  the  N-version  software.  This  value 
might  be  subtracted  from  the  transfer  function  of  tne  N-version  software 
block  to  produce  a  conservative  value  (minimum)  of  the  »*e  T  i  a  d  i  1  i  ty  (or 
safety  or  availability)  of  the  N-version  software  and  consecuently  a 
conservative  estimate  (minimum)  of  the  overall  software  reliability  value 
(or  safety  or  availability  value). 

The  following  equation  gives  the  maximum  probability  of  coincident 
errors  (E). 

E  =  (z)*(l  -  GL)Z  +(z+l)*  (1  -  GL)Z  +  1  + 

(z+2)*  (1  -  Gl)Z+2  +  ....  +(n)*  (1  -  GL)n 

with  n  =  the  number  of  versions  in  the  N-version  software; 

G-j  =  the  reliability  (or  safety  or  availability)  value  for 
version  i ; 

i  =  1 ,  2, . . .n; 

G[_  =  the  largest  reliability  (or  safety  or  availability)  value 
among  the  group  of  r  versions  being  evaluated; 

z  -  (n/2)  if  n  is  an  even  number; 

z  =  [(n  +  l)/2]  if  n  is  an  odd  number; 

(r)  =  the  number  of  r  combinations  of  an  n  element  set;  and 

(ry*  =  the  actual  r  combinations  of  (1  -  G-j)  values  for  the 
different  versions  in  an  n  element  set  of  versions. 

(See  Appendix  I  for  some  examples  which  consider  the  effect  of  coincident 
errors. ) 

8.0  ASSUMPTIONS 

7-e  assumptions  that  are  frequently  made  with  the  various  software 
irss'-'oec  below,  along  with  the  reasons  for  such  assumptions. 

■-  :  a"-?  logically  grouped  below  the  corresponding  software 
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to  benefit  the  reader.  In  the  development  of  this  software  reliability 
model,  it  is  assumeo  that  the  software  of  the  function  blocks  will  have 
complete  probabilistic  i ndepencence .  However,  Section  7.0  tries  to  accommodate 
for  any  ill  effects  that  result  from  this  basic  assumption. 


-‘oN 

-N-C- 

*  i 

<Vi 

sV* 


Sinale  Version  Software 


a.  Errors  are  not  always  corrected  when  detected  and  errors 
may  be  spawned  when  correcting  errors. 

b.  The  time  to  remove  a  failure  is  considered  to  be  negligible 
and  is  ignored. 

c.  Inputs  which  exercise  the  program  are  randomly  selected. 


r,V 


d.  The  failure  rate  at  any  time  is  proportional  to  the  current 
number  of  errors  remaining  in  the  program  [PRATER]. 


N- Vers  ion  Software 


a.  To  benefit  from  increased  reliability,  N-version  assumes 
the  probability  of  a  common  fault  among  the  versions  is 
extremely  low. 

b.  When  a  fault  is  determined,  the  damage  incurred  is  limited 
to  the  encapsulation  of  the  individual  software  versions 
and  the  overall  function  that  the  versions  are  performing. 

Decision  Algorithm 


a.  For  a  majority  vote,  it  is  assumed  that  damage  will  be 
limited  to  the  versions  in  the  minority  when  the  decision 
algorithm  is  invoked. 

b.  It  is  possible  for  a  majority  vote  to  yield  an  incorrect 
result  if  a  majority  of  the  inputs  are  incorrect. 


Recovery  Block 

a.  Faults  will  manifest  themselves  within  a  recovery  region. 

b.  The  alternate  versions  of  software  components  are  independent 
sucn  that  correlated  faults  are  either  eliminated  or  reduced 
to  an  acceptably  low  level. 


c.  The  n  alternate  blocks  are  independent  from  the  acceptance 
tests  [HITT84]. 

Acceptance  Test 

a.  The  acceptance  test  will  recognize  the  faults. 


.v 

A  »*, 

-\-v 
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APPENDIX  I 

N-VERSION  SOFTWARE  CALCULATIONS 

The  transfer  function  for  N-version  software  is  given  by  the 
equation : 

1 

Cn 

1  -  n  ( i— i ) 
i  =  i 

n  =  the  number'  of  versions  in  the  N-version  software; 

Cn  =  C(n,z)  =  the  number  of  z  combinations  of  the  n 
element  set; 

=  the  product  of  rel iabi 1 i ties  of  the  i-th  combination 
required  for  success;  and 

i  =  1,  2, . . . .Cn. 

z  =  [ ( n/2 )  +  1]  if  n  is  an  even  number 
z  =  [(n+l)/2]  if  n  is  an  odd  number 

The  following  examples  utilize  this  equation. 

rxample  1 

Using  the  block  diagram  shown  in  Figure  1,  a  system  with  N-version 
software  whose  outputs  go  into  a  decision  algorithm  will  be  analyzed. 

For  this  example,  the  N-version  software  will  consist  of  five  versions. 

The  reliability  values  for  the  five  versions  and  the  decision  algorithm 
are  given  in  Table  5. 


Software  Component 

Reliability  Value 

Version  1 

0.77 

Version  2 

0.82 

Version  3 

0.65 

Version  4 

0.91 

Version  5 

0.89 

Decision  Algorithm 

0.997 

Table  5.  Reliability  Values  for  the  Software  Components 
in  the  Figure  that  Represents  the  General 
Format  for  N-Version  Software 


fol 1 owi ng 


wi  th 


To  determine  the  overall  software  reliability  value  for  this 
block  diagram,  the  transfer  function  for  the  N-version  software,  which 
is  dependent  upon  the  number  of  versions  (in  this  example  n  =  5),  must 
first  be  determined.  Hence,  the  transfer  function  for  the  N-version 
software  (NVS)  is 

NVS  =  [1  -  ( 1-G1G2G3) ( 1-G1G2G4) ( 1-GiG2G5 ) ( 1-G1G3G4) ( I-G1G3G5) 

( 1-G1G4G5)( i-g2g3g4)( 1-G2G3G5) ( 1-G2G4G5)  ( 1-G3G4G5)] 

By  substituting  in  the  appropriate  reliability  values, 

NVS  =  1  -  [ 1- (0. 77 ) (0. 82 )(0.65)][1- (0.77) (0.82) (0.91)]  x 

[l-(0.77 ) (0.82) (0.89)][l-(0.77 ) (0. 65) (0. 91)3  x 
[l-(0.77 ) (0.65)(0.89)][1— (0.77) (0.91) (0.89)]  x 
[1-(0.82)(0.65)(0.91)][1-(0.P2) (0.65) (0.89)]  x 
[1-(0.82)(0.91)(0.89)][1- (0.65) (0.91) (0.89)] 

=  [1  -  ( l-0.41041)(l-0. 574574)( 1-0. 561946) ( 1-0.455455)  x 
(1-0. 445445) (1-0. 623623) (1-0. 48503) (1-0. 47437)  x 
(1-0. 664118) ( 1-0. 526435)] 

=  [1  -  (0. 58959) (0. 425426 ) (0.438054) (0. 544545) (0. 554555)  x 
(0.376377) (0.51497) (0.52563) (0.335882) (0.473565)] 

=  1  -  0.0005377  =  0.9994623 

NVS  =  0.99946 

By  applying  the  software  reliability  model  in  Section  4,5, 
through  Ln  =  0 

Gi  =  (0.99946)  x  (0.997)  =  0.9964616 

G?  through  G|<  =  0 


Therefore , 


(0.9964616)  x  (1) 

(1) 


Rel iaoil i ty  =  — 
Reliability  =  0.996. 


-  =  0.9964616 


The  probability  of  coincident  errors  (E)  should  be  determined 
and  subtracted  from  the  transfer  function  for  the  N-version  software 
to  improve  the  accuracy  of  the  reliability  value  of  the  N-version  software 
and  the  accuracy  of  the  overall  software  reliability  value.  (Reference 
Section  7.0.)  For  this  example, 


E 


(1-GL)4  + 


(1-GL)5. 


The  groups  for 


would  be 


G1G2G3,  G1G2G4,  G1G2G5 .  G1G3G4,  G1G3G5,  G1G4G5,  G2G3G4, 
G2G3G5,  G2G4G5,  and  G3G4G5, 

with  respective  G[_  values  being 

G2,  G4,  G5,  G4,  G5,  G4,  G4,  G5,  G4,  and  G4. 

The  G|_  values  can  be  grouped  as  G2  +  6  x  G4  +  3  x  G5. 

(  5> 

The  groups  for\4/  would  be 


G^G2G3G4>  G1G2G3G5,  G1G2G4G5,  G1G3G4G5,  and  G2G3G4G5, 
with  the  respective  G|_  values  being  G4,  G5,  G4,  G4,  and  64.  This  gives 
4  x  G4  +  G5. 

(5y 

The  group  for  \  5/  is  G1G2G3G4G5  with  G|_  =  G4. 


Therefore , 


E  =  (l-G?)3  +  6  x  ( 1  -  G  4 )  ^  +  3  x  ( 1  -  G  5 )  +  4  x  (l-Ga)4  + 

(1-G5)d  +  (1-G4)5 

■  ( 1-0.82) 3  +  6  x  ( 1-0.91) 3  +  3  x  ( 1-0.89)3  +  4  x  (1-0. 91)4  + 
( 1-0.89)4  +  ( 1-0 . 91 ) 5 

=  (0. 18) 3  +  6  x  (0.09)3  +  3  x  ( 0 . 1 1 ) 3  +  4  x  (0.09)4  + 

(0.11)4  -  (0.09)5 

=  0.005332  -  0.004374  +  0.003993  +  0.0002624  +  0.0001464  + 
0.0000059 
E  =  0 .0146137 . 


Subtracting  E  from  the  transfer  function  for  the  N-version  software  gives 

NVS  =  0.9994623  -  0.0146137  =  0.9848486 
NVS  =  0.98485. 


L]_  through  Ln  =  0 

Gi  =  (0.98485)  x  (0.997)  =  0.9818955 
G2  through  G«  =  0 
41  =  1 
4  =  1 

Hence,  the  adjusted  Reliability  value  is 

Reliability  =  (°-9818955)  x  (*)  =  0.9818955 

(1) 

Reliability  =  0.982. 


Example  2 

This  example  will  evaluate  the  overall  reliability  for  the 
system  shown  in  Figure  2.  In  this  example,  the  N-version  software  will 
consist  of  four  versions.  Each  of  the  outputs  from  the  N-version  software 
are  submitted  to  an  acceptance  test  (the  identical  acceptance  test  is 
used  for  all  four  versions),  and  then  the  outputs  from  the  acceptance 


test  are  input  to  the  decision  algorithm.  The  reliability  values  for 
each  of  the  software  components  are  given  in  Table  6. 


Software  Component 

Re  1  iabi 1 i ty  Va 1 ue 

Version  1 

0.86 

Version  2 

0.79 

Version  3 

0.94 

Version  4 

0.63 

Acceptance  Test 

0.98 

Decision  Algorithm 

0.93 

Table  6.  Reliability  Values  for  the  Software  Components 
in  the  Figure  that  Represents  the  N- Version 
Software  with  Acceptance  Tests 

First,  the  transfer  function  for  the  N-version  software  (NVS) 
must  be  determined.  In  this  example, 


NVS  =  [1  -  ( 1-G3.G2G3) (1-GiG3G4)(  1-G1G2G4)(  1-G2G3G4)]. 


By  substituting  in  the  appropriate  reliability  values. 


NVS  =  {1  -  [1  -  (0.86) (0.79) (0.94)][1  -  (0.86) (0. 94) (0. 68) ]  x 
[1  -  (0.86) (0.79) (0.68)][1  -  (0. 79) (0. 94) (0. 68)] } 

=  [1  -  (1-0. 638636) ( 1-0. 549712) ( 1-0 . 461992 )( 1-0 . 504968)] 

=  [1  -  (0. 361364) (0. 450288) (0. 538008) (0. 495032)] 

=  1  -  0.0433368  =  0.9566632 
NVS  =  0.95666. 


By  applying  the  software  reliability  model  in  Section  4.5, 


l_i  through  Ln  =  0 

Gi  =  (0.95666)  x  (0.98)  x  (0.93)  =  0.8718999 
G2  through  G«  =  0 


irere  ore, 


Reliability  =  (0-S718999)  x  ( U_  =  o. 8718^99 

(i) 

Reliability  =  0.872. 


The  probability  of  coincident  errors  (E)  for  this  example  is 


-  (r- 

C  =  V  £.• 


(i-gl)4  +  VzT  (i-gl)3  +  (4)*  (i-gl)4. 


( -> 

ne  groups  ror  \  2)  are 


G xG 2 »  G 1 G 3 ,  G1G4,  G2G3 ,  G2G4,  and  G3G4. 

Their  respective  G|_  values  are  Gx ,  G3,  Gi,  G3,  83,  and  G3.  These  values 
can  be  grouped  as  2  x  G^  +  2  x  G2  +  2  x  G3. 

The  groups  for  (\y  are  G1G2G3,  GiG2G4»  G1G3G4,  and  G2G3G4, 
with  the  Gl  values  G 3 ,  Gj,  G3,  and  G 3 ,  respectively.  This  gives  Gj  + 

3  x  G3. 

f4V 

The  group  for  \4/  is  G1G2G3G4  with  Gl  =  G3. 

Hence , 

E  =  2  x  (1-Gi)2  +  2  x  (1-G2)2  +  2  x  (1-G3)2  +  ( 1-Gx )3  + 

3  x  (1-G3)3  +  (1-G3)4 

=  2  x  (  1-0. 86)2  +  2  x  (  1-0.79)2  +  2  x  (  1-0.94)2  +  (  1-0.86)3 
+  3  x  (1-0. 94)3  +  (1-0. 94)4 

=  2  x  (0.14)2  +  2  x  ( 0 . 2 1 ) 2  +  2  x  (0.06)2  +  (0.14)3  +  3  x 

( 0 . 06 ) 3  +  ( 0 . 0  6 ) 4 

=  0.0392  +  0.0882  +  0.0072  +  0.002744  +  0  000648  +  0.00001296 
E  =  0.1380049. 


Re-evaluating  the  reliability  value  for  the  N-version  software  and  the 
overall  software  reliability  value  give 


NVS  =  0.9556632  -  0.00001296  =  0.9566503 
NVS  =  0.95665 
Li  through  Ln  =  0 

Gi  =  (0.95665)  x  (0.98)  x  (0.93)  =  0.8718908 
G2  through  G«  =  0 
A1  =  1 
A  =  1 

Tnerefore,  the  adjusted  reliability  value  is 

Reliability  =  87 18908  x  1  =  0.8718908 

1 

Reliability  =  0.872. 


Example  3. 

This  example  will  be  more  complicated,  involving  N-version 
software  in  which  only  x  verions  are  used  at  a  time  (reference  Figure 
3).  For  this  example,  the  number  of  versions  in  the  N-version  software 
will  be  nine.  Three  of  the  versions  will  be  run  at  a  time,  and  their 
outputs  sent  to  the  decision  algorithm.  If  the  decision  algorithm  fails, 
then  the  system  will  rollback,  and  the  next  three  versions  will  be  run. 
This  cycle  will  continue  until  the  decision  algorithm  passes  or  until 
all  of  the  versions  in  the  N-version  software  have  been  run.  Table  7 
gives  the  reliability  values  for  each  of  the  components  in  this  example. 

First,  the  transfer  function  for  the  N-version  software  in 
which  only  x  versions  are  used  at  a  time  (NVSx)  must  be  determined. 

The  transfer  function  is  a  combination  of  the  equations  in  Section  4.5 
for  N-version  software  and  a  recovery  block  since  the  usage  of  x  versions 
at  a  time  is  N-version  software,  but  applying  rollback  and  going  through 
another  x  versions  incorporates  the  concept  of  a  recovery  block.  Hence, 
the  transfer  function  for  the  N-version  software  in  which  only  x  versions 
are  used  at  a  time  is 


■pw are  Component 


Version 

1 

Version 

2 

Version 

3 

Versi on 

4 

Versi on 

5 

Version 

6 

Ve’-si  on 

7 

Versi on 

8 

Version 

S 

sion  Algorithm 

Rollback 

Table  7.  Reliability  Values  for  the  Software  Components 
in  the  Figure  that  Represents  the  N-Version 
Software  in  Which  Only  x  Versions  are  Used 
at  a  Time 


NVSx  =  [1  -  (1-G1G2)(2-G1G3)U-G2G3)]  + 

| 1-[1-(1-GiG2)(1-G1G3)(1-G2G3)]]  x 

[1- ( I-G4G5) ( 1-G4G6 ) ( I-G5G5 ) ]  + 

1  1-[1-(1-G1G2)(1-G1G3)(1-G2G3)]}  x 
{ 1- [ 1- ( I-G4G5 ) ( 1- G4G6 )( I-G5G5)]}  X 
[  1- ( 1-G7G8) ( I-G7G9) ( 1— G8G9 ) ] . 

By  substituting  in  the  appropriate  reliability  values  for  this  example, 

NVSx  =  {l-[l-(0.84)(0.71)][l-(0.84)(0.66)][l-(0.71)(0.66)]j 

+  [1-  {1-[1- (0.84)  (0.71)] [1- (0.84) (0.66)] [1- (0.71) (0.66)];  ] 
x  { l-[l-(0. 87) (0.92)] [l-(0.87) (0.90)] [1- (0.92) (0.90)]! 

+  [l-{l-[l-(0.84) (0.71 )][l-(0.84)(0.66)][l-(0. 71) (0.66)]!  ] 
x  [  1  - ]  1  - [  1  - ( 0  87 ) (0. 92)][ 1- (0. 87 ) (0. 90) ] [ l-(0 . 92) (0. 90) ] j  ] 
x  ‘  1-[1- (0.73) (0.85)] Li- (0.73) (0.78)1[1- (0,85) (0.78)1 1 


4 


=  [1- ( 1-0.5954) ( i-0. 5544) ( 1-0.4686)1  +  i l-[ i- ( 1-0 . 5964)  x 
( i-0. 5544) ( 1-0 . 4686 ) j ■  x  [1- { 1-0. 8004) ( 1-0. 783) ( 1-0.828 ) ] 

+  - l-[l-( 1-0. 5964) (1-0. 55^4) ( 1-0.4636)] -  x  j 1- [l-( 1-0. 8004)  x 
( 1-0.783) ( 1-0.828)] •  x  [l-( 1-0.6205) ( 1-0. 5694) ( 1-0.663) j 

=  [l-( 0.40 36) (0.44 55) (0.5214)]  +  { l-[ 1- (0 . 4026 ) (0 . 4456)  x 
(0.5314)]-  x  [l-(0. 1S96 )(0.217)(0.172)]  +  { l-[l-0. 4026)  x 
(0.4456)  (0. 5314)3}  x  \  1- [  1-  (0. 1996)  (0 . 217 )  (0. 172 )] 1  x 
[ 1- ( 0 . 37  95 ) ( 0 . 4306) (0. 337 ) j 

=  (1-0.095569187)  +  [l-( 1-0.095569187) j  x  (1-0.0074498704) 

+  [l-(l-0. 095569187)]  x  [1- ( 1-0. 0074498704)]  x  (1-0.0550701) 

=  0.9044309  +  ( 1-0 . 9044309) (0.9925501)  +  (1-0.9044309)  x 
(1-0. 9925501) (0. 9449299) 

=  0.9044309  +  0.0948571  +  0.000672771 
NVSx  =  0.9999608 

By  applying  the  software  reliability  model, 

L}  =  (0.9999608)  x  (0.91)  x  (1-0.99)  =  0.009099643 
[The  transfer  function  for  rollback  is  (1.0  -  reliability  value).  (Reference 
Section  4.5.)] 

I_2  through  Ln  =  0 

Gi  =  (0.9999608)  x  (0.91)  =  0.9099643 
63  through  Gi<;  =  0 

A1  =  1 

A  =  1  -  0.009099643  =  0.9909004 

Therefore , 

Reliability  *  (0 .9099643 ;)_x  (J_).  =  0. 9183207 

(0.9909004) 

Reliability  =  0.918. 


The  accuracy  of  the  hybriG  N-version  software  reliability  value 
a*fectec  if  net  all  of  the  n  versions  are  usea.  (Reference  Sectio 
For  this  examcie,  it  is  assumed  that  only  six  of  the  nine  version 
jally  usee.  Inis  gives 

NVSx  =  [1- ( 1 —  3  n  G  2 ) ( I-G1G3) ( I-GnGg) j  + 

• l-[i-( l-GjGo) ( l-GiGg) ( I-G2G3) ]}  x 
[  1- { 1-Gi3c ) ( I-G4G5) ( 1 - G  5  G  5 ) ] 

=  0 . SC-— 3CS  ♦  0. 09*8571 

NVSx  =  0.999288 

Ll  =  (0.999288)  x  (0,91)  x  (1-0.99)  -  0.009093521 
l_2  through  Ln  =  0 

G]  =  (0.999288)  x  (0.91)  =  0.90935208 
G2  through  G«  =  0 
*1  =  1 

a  =  1-0.0090935  =  0.9909065 
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The  probability  of  errors  for  the  first  group  of  three  versions 


E !  =  (!)*  (1-GL)2  +(3)*  (1-GL)3. 


The  groups  for \2J  are  G1G2.  6xG3,  and  G2G3,  with  GL  =  2  x  Gx  +  G2. 
The  group  for  \3/  is  G1G2G3  with  Gj_  =  Gj. 

Hence , 


=  2  x  (1-Gi)2  +  (1-G2)2  +  (1-GO3 
=  2  x  ( 1-0.S4)2  +  ( 1-0.71)2  +  ( 1-0.84)3 
=  0.0512  +  0.0841  +  0.004096 
=  0.139396. 


The  probability  of  errors  for  the  second  group  of  three  versions 


E2  =  (D*  (1-GL)2  +(3)*  (1-GL)3. 

r3v 

The  groups  for  V?/  are  G4G5>  G4G6»  and  G5G6,  with  G[_  =  2  x  G5  +  Gs. 
The  group  for  \ 3y  is  G4G5G5  with  Gl  =  G5. 

The  value  for  E2  is 

E2  =  2  x  (1-G5)2  +  (1-G6)2  +  (1-G5)3 

=  2  x  (1-0. 92)2  +  (1-0. 90)2  +  (1-0. 92)3 
=  0.0128  +  0.01  +  0.000512 
E2  =  0.023312. 


The  probability  of  errors  for  the  third  group  of  three  versions 


e3  -(2)*  (1-GL)2  +(3)*  (1-GL)3. 


•V 
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( 3> 

The  groups  of  \2/  for  E3  are  G7G3,  G7G9,  and  GgGg,  with  G|_  =  2  x  Gg 
+  Gg.  The  group  of  (3)  is  G7G3G9  with  G[_  =  Gg. 

Hence, 

E3  =  2  x  ( 1-0.85)2  +  ( 1-0. 78) 2  +  ( 1-0.S5)3 
=  0.04E  +  0.0484  +  0.003375 

E3  =  0.096775. 

Considering  these  probabilities  when  the  transfer  function 
for  the  N-version  software  (in  which  only  x  versions  are  used  at  a  time) 
is  calculated  gives 

NVSx  =  [l-( l-G^Gg) ( 1-GiG3)( 1-G2G3)-Ei]  + 

{1-[1-(1-G1G2)(1-G1G3)(1-G2G3)-E1]}  x 
[l-( I-G4G5) ( 1-G4G6) ( 1-G5G6)-E2]  + 
(1-[1-(1-G1G2)(1-GiG3)(1-G2G3)-E1]}  x 
{l-[l-( I-G4G5 ) ( 1-G4G6)(1-G5G6)-E2] }  x 
[l-( I-G7G3) ( I-G7G9) ( I-G8G9 )-e3] 

=  (1-0.095569187  -  E^  +  [l-(  1-0.095569187  -  E^J  x 
(1-0.0074498704  -  E2)  +  [l-( 1-0.095569187  -  Ei)]  x 
Cl-( 1-0.0074498704  -  E2)]  x  (1-0.0550701  -  E3) 

=  (0.9044309  -  Ei)  +  (1-0.9044309  +  Ex) (0.9925501  -  E2) 

+  (1-0.9044309  +  Ei) ( 1-0. 9925501  -  E2) (0. 9449299  -  E3) 

=  (0.9044309  -  0.139396)  +  (0.0955691  +  0, 139396) (0 . 9925501 
-  0.023312)  +  (0.0955691  +  0. 139396) (0. 0074499  +  0.023312) 
x  (0.9449299  -  0.096775) 

=  0.7650349  +  (0. 2349651 ) (0. 9692381 )  +  (0. 2349651 ) (0. 0307619) 
x  (0.8481549) 

=  0.7650349  +  0.2277371  +  0.0061304 

NVSx  =  0.9989024. 
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The  overall  software  reliability  value  with  the  coincident  errors  considered 
is  determined  as 


L],  =  (0.9989024)  x  (0.91)  x  (1-0.99)  =  0.Q0909C0118 
1-2  through  Ln  =  0 

Gi  =  (0.9989024)  x  (0.91)  =  0.9090012 
G2  through  G«  =  0 
Ai  =  1 

4=1-  0.0090900118  =  0.99091 


Thereto  re. 


Reliability  =  (0-9090012)  x  (_LL  ,  0. 9173398 


(0.99091) 
Rel iabil ity  =  0.9173. 


Example  4. 

The  hybrid  N- version  software  format,  shown  in  Figure  4,  will 
be  evaluated  in  this  example.  In  this  example,  the  N-version  software 
will  consist  of  three  versions.  The  outputs  of  these  versions  are  fed 
into  a  decision  algorithm.  If  the  decision  algorithm  fails,  then  the 
software  will  rollback  and  run  through  the  three  versions  again.  However, 
this  time  the  outputs  of  the  versions  are  input  to  an  acceptance  test 
prior  to  entry  to  the  decision  algorithm.  For  this  example,"  it  is  assumed 
that  the  reliability  values  for  the  individual  software  components  are 
as  given  in  Table  8. 

The  transfer  function  for  the  N-version  software  is 
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W. 


A-i 

K 

.V. 


NVS  =  1-(1-GiG2)( 1-GxG3) ( 1-G2G3) 

=  1-[1- (0.67) (0.78)] [1- (0.67) (0.89)] [1- (0.78) (0.89)] 
=  l-( 1-0. 5226) (1-0. 5963) (1-0. 6942) 

=  l-(0. 4774) (0. 4037 ) (0. 3058 ) 

NVS  =  0.94106428. 


vvv 


I 


fi 


Software  Component 

Reliability  Value 

Version  1 

0.67 

Version  2 

0.78 

Version  3 

0.89 

Acceptance  Test 

0.86 

Decision  Algorithm 

0.88 

Rol 1  back 

0.98 

Table  8.  Reliability  Values  for  the  Software  Components 
in  the  Figure  that  Represents  the  N-Version 
Software  in  Which  the  Outputs  are  Subjected  to 
an  Acceptance  Test  if  the  Decision  Algorithm 
Fails 

The  variables  of  the  software  reliability  model  are 

Closed  Loop  #1  =  (0.94106428)  x  (0.88)  x  (1-0.98)  =  0.01656273 
Closed  Loop  #2  =  (0.94106428)  x  (1-0.86)  x  (0.88)  x  (1-0.98) 

=  0.0023187824 

[Remember  that  the  transfer  function  of  the  equivalent  block  in  a  feedback 
or  feed-forward  path  is  (1.0  -  reliability  value).] 

Ill  =  Closed  Loop  #1  +  Closed  Loop  #2  =  0.018881512 
£L2  through  ILn  =  0 

G]_  =  (0.94106428)  x  (0.88)  =  0.82813657 

G2  =  (0.94106428)  x  (1-0.86)  x  (0.88)  =  0.11593912 

G3  through  G|(  =  0 

Ai  =  1 

a2  =  1 

A3  through  A«  =  0 

A  «  1  -  ILj  »  1  -  0.018881512  =  0.981118488 


Therefore , 


Reliability  =  (0-82813657  ,  1)  *  (0.1159391;  »  1)  .  „  ,9622444 

(0.981118408) 

Reliability  =  0.962. 
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To  improve  the  accuracy  of  the  software  reliability  model, 
the  probability  of  coincident  errors  (E)  might  be  considered.  (Reference 
Section  7.0.)  For  this  example, 

E  =  (2)*  (1-GL)2  +  (3)*  (  1-Gl)3. 

(  3\* 

The  groups  for  \ZJ  are  GiG2,  G 1G3 ,  and  G2G3  with  resoective  Gl  values 
of  G2.  G3,  and  G3,  or  G2  +  2  x  63.  The  group  for  (u)  is  G1G2G3  with 
GL  -  S3. 

Thus , 

E  =  (1-G2)2  +  2  x  (1-G3)2  +  (I-G3)3 
=  ( 1-0.7S)2  +  2  x  ( 1-0.89)2  +  (1-0. 89)3 
=  ( 0 . 22 ) 2  +  2  x  (0.11)2  +  (0.11)3 
=  0.0484  +  0.0242  +  0.001331 
E  =  0.073931. 

By  subtracting  the  probability  of  coincident  errors  from  the  N-version 
software  transfer  function,  a  conservative  value  of  the  reliability  value 
for  the  N-version  software  and  the  overall  software  reliability  value 
can  be  determined. 

NVS  =  0.94106428  -  0.073931  =  0.86713328 

Closed  Loop  #1  =  (0.86713328)  x  (0.88)  x  (1-0.98)  =  0.015261546 
Closed  Loop  #2  =  (0.86713328)  x  (1-0.86)  x  (0.88)  x  (1-0.98) 

=  0.0021366164 

LL 1  =  0.015261546  +  0.0021366164  =  0.017398162- 
EL2  through  ELn  =  0 

Gi  =  (0.86713328)  x  (0.88)  =  0.76307729 

G2  =  (0.86713328)  x  (1-0.86)  x  (0.88)  =  0.10683082 

G3  through  G«  =  0 

41  =  1 

A2  =  1 

A3  through  A|<  =  0 

A  =  1  -  r L 1  =  1  -  0.017398162  =  0.98260184 


Therefore , 


Re  1  iabil i ty  = 


(0.76307729  x  1)  +  (0.10683082  x  1) 
(0.98260184) 


=  0.88531089 


Re  1 i a  D i 1 ity  =  0.885. 


APPENDIX  II 

RECOVERY  BLOCK  CALCULATIONS 


The  transfer  function  for  a  recovery  block  is  dependent  upon  the 
number  of  alternates  (n)  that  are  used.  This  transfer  function  is  calculated 
with  the  following  equation: 

Si  +  (1  •  G1JG2  +  (1  -  Gi)(l  -  G2 ) G3  +  .... 
with  Gi  =  the  reliability  value  for  alternate  i  and 
i  =  1 ,  2 ,  3 , . . .  n . 

The  examples  below  demonstrate  the  determination  of  the  overall  software 
reliability  value  with  this  equation  and  the  software  reliability  model. 

Example  1 

Figure  5  shows  the  general  format  of  a  backward  recovery  block. 

For  this  example,  the  number  of  alternates  will  be  four.  The  reliability 
value  for  each  of  the  software  components  is  listed  in  Table  9. 


Software  Component 

Reliability  Value 

Alternate  1 

0.86 

Alternate  2 

0.75 

Alternate  3 

0.79 

Alternate  4 

0.84 

Acceptance  Test 

0.91 

Rol 1  back 

0.93 

Table  9.  Reliability  Values  for  the  Software  C< 
in  the  Figure  that  Represents  the  Gent 
Format  of  a  Backward  Recovery  Block 


The  transfer  function  for  the  recovery  block  (RB)  is 


RB  =  Gi  +  (1  -  Gi)G2  +  (1  -  G]J(1  -  G2) G3  + 
(1  -  Gi)(l  -  G2) ( 1  -  G3)G4 


=  0.86  +  (  1  -  0.86) (0.75)  +  (1  -  0.86) (  1  -  0.75) (0 . 79 
(1  -  0.86) ( 1  -  0.75) ( 1  -  0.79) (0.84) 

=  0.86  +  (0. 14) (0.75)  +  (0. 14) (0. 25) (0.79)  + 

(0. 1^) (0.25) (0.21) (0.84) 

=  0.86  +  0.105  +  0.02765  +  0.006174 
RB  =  0.998824 


The  variables  of  the  software  reliability  model  will  be 

Ll  =  (0.998824)  x  (0.91)  x  (1.0  -  0.93)  =  0.0636251 

[Recall  that  the  transfer  function  for  rollback  is  (1.0  -  reliabil 
This  was  defined  as  such  in  Section  4.5.] 

1-2  through  ln  =  0 

Gi  =  (0.998824)  x  (0.91)  =  0.9089298 
G3  through  G«  =  0 
Aj  =  1 

A2  through  ak  =  0 
A  =  1  -  0.0636251  =  0.9363749 


Therefore , 


Re  1 iabi 1 i ty 


(0.9089298)  x  (1) 
(0.9363749) 


0.9706901 


i ty  value) 


Re  1 iabi 1 i ty  =  0.97. 
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As  was  discussed  in  Section  6.1.2,  if  only  two  of  the  alternates 
are  actually  used,  although  the  recovery  block  supplies  four  alternates, 
this  will  decrease  the  reliability  of  the  recovery  block  and  consequently 
decrease  the  overall  software  reliability.  The  following  calculations  show 
this. 

RB  =  Gx  +  (1  -  Gi )G2  =  0.86  +  (1  -  0.86) (0.75) 


=  0.86  +  0.105  =  0.965 

Ll  =  (0.965)  x  (0.91)  x  (1  -  0.93)  =  0.0614705 
1-2  through  Ln  =  0 
Gi  =  (0.965)  x  (0.91)  =  0.9089298 
G2  through  G<  =  0 


A1  =  1 

through  Aj(  =  0 

A  =  1  -  0.0614705  =  0.9385295 


Hence, 


Reliability  =  . ,(.0.9089298)  x  (1)..  =  q. 9684616 
(0.9385295) 


Reliability  =  0.968  when  only  two  of  the  alternates  are  used. 


Example  2 

The  general  format  of  a  forward  recovery  block  is  shown  in  Figure 
6.  This  example  will  evaluate  the  overall  software  reliability  of  this  figure 
(six  alternates  will  be  used:  one  primary  alternate  and  five  additional 
alternates),  with  the  reliability  values  assigned  as  shown  in  Table  10. 

The  transfer  function  for  the  recovery  block  (RB)  is 

RB  =  Gi  +  (1  -  Gi )G2  +  (1  -  Gi ) ( 1  -  G2)G3  + 

(1  -  G ! ) ( 1  -  G2) ( 1  -  G3)G4  + 

(1  -  GX) ( 1  -  G2)(l  -  G3)(l  -  G4)G5  + 

(1  -  G i ) ( 1  -  G2)(l  -  G3)(l  -  G4) ( 1  -  G5)G6 
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Software  Component 

Reliability  Value 

Alternate  1 

0.81 

Alternate  2 

0.72 

Alternate  3 

0.73 

Alternate  4 

0.74 

Alternate  5 

0.85 

Alternate  6 

0.86 

Acceotance  Test 

0.97 

Rol 1  back 

0.98 

Rol 1 -Forwa ra 

0,89 

Table  10.  Reliability  Values  for  the  Software  Components 

in  the  Figure  that  Represents  the  General 
Format  of  a  Forward  Recovery  Slock 

=  0.81  +  (1  -  0.81) (0.72)  +  (1  -  0.81) ( 1  -0.72) (0.73)  + 

(1  -  0.81) ( 1  -  0. 72) ( 1  -  0.73)(0.74)  + 

(1  -  0.81) ( 1  -  0.72) ( 1  -  0. 73) ( 1  -  0.74) (0.85)  + 

(1  -  0. 81 ) ( 1  -  0.72)(1  -  0.73)(1  -  0.74)(1  -  0.85)(0.S6) 

=  0.81  +  (0. 19) (0.72)  +  (0. 19) (0. 28) (0. 73)  + 

(0.19)(0.28)(0.27)(0.74)  +  (0. 19) (0. 28) (0. 27) (0. 26) (0.85) + 

(0.19) (0.28) (0.27) (0.26) (0.15) (0.86) 

=  0.81  +  0.1368  +  0.038836  +  0.01062936  +  0.003174444  +  0.00048176856 

=  0.99992157256 
RB  =  0.9999216. 


With  the  software  reliability  model, 

Li  =  (0.9999216)  x  (0.97)  x  (1  -  0.98)  =  0.019398479 
[Note  that  the  transfer  function  for  rollback  is  (1.0  -  reliability  value).] 


I_2  through  Ln  =  0 

Gj  =  (0.9999216)  x  (0.97)  =  0.96992395 

G2  »  (0.9999216)  x  (0.97)  x  (1  -  0.98)  x  (1  -  0.89)  =  0.0021338327 


[Remember  that  the  transfer  function  for  rollback  and  roll-forwara  are  (l.C 
-  reliability  value).  This  was  discussed  in  Section  4.5.] 

G3  through  G«  =  0 
Al  =  1 
A2  =  1 

A3  through  ak  =  0 
A  =  1  -  0.019298479  =  0.98060152 


Therefore , 


Rel iabil ity  = 


(0.96992395  x  1)  +  (0.0021338327  x  1) 


(0.98060152) 

=  (0.97205778)/(0. 98060152)  =  0.99128725 
Rel iabil ity  =  0.991. 


Discussion  of  the  Results: 


This  result  is  as  expected.  With  just  the  n  alternates,  acceptance 
test,  and  rollback,  the  overall  reliability  would  be 


Reliability 


(0.9999216)(0.97) _ 

1  -  (0.9999216) (0. 97 ) ( 1  -  0.98) 


Reliability  =  0.9891112  =  0.989. 


The  roll-forward  should  increase  this  reliability  value,  as  it  does. 

To  evaluate  the  effect  on  accuracy  when  less  than  the  n  alternates 
(in  this  example  n  =  6)  are  actually  used,  the  reliability  of  this  example 
will  be  evaluated  with  n  =  3,  n  =  4,  and  n  =  5. 

For  n  =  3, 

RB  =  Gi  +  (1  -  GX )G2  +  (1  -  Gj ) ( 1  -  G2)G3 

=  0.81  +  (  1  -  0.81)(0.72)  +  (  1  -  0.81) (  1  -  0.72) (0.73) 

=  0.81  +  0.1368  +  0.038836 


RB  =  0.985636 


Using  the  software  reliability  model, 

Ll  =  (0.985636 )  (0. 97 ) (  1  -  0.98)  =  0.019121338 
1-2  through  Ln  =  0 

Sj  =  (0. 985636) (0.97)  =  0.95605692 

G2  =  (0. 985636) (0 . 97 ) ( 1  -  0 . 98 ) ( 1  -  0.89)  =  0.0021033472 
G3  through  G«  =  0 
41  =  1 
-2  =  1 

~2  through  4^  =  0 

4=1-  0.019121338  =  0.98087866 

gives  Reliability  =  (0-95606692  x  1)  +  (0.0021033472_  _x  _1)  _  g  Q7684893 

(0.98087866) 

Rel iabil ity  =  0.977. 

For  n  =  4, 

RB  =  0.81  +  0.1368  +  0.038836  +  0.01062936 
RB  =  0.99626536 

Using  the  software  reliability  model, 

Li  =  (0.99626536) (0.97) ( 1  -  0.98)  =  0.019327547 
l_2  through  Ln  =  0 

G\  =  (0.99626536) (0.97)  =  0.9663774 

G2  =  (0.99626536) (0. 97) ( 1  -  0 . 98 ) ( 1  -  0.89)  =  0.0021260303 
G3  through  G«  =  0 

41  *  1 

42  «  1 

43  through  4|<  =  0 

4=1-  0.019327547  =  0.98C67245 

gives  Reliability  •  J°,^63774  x  1)  *  (0. 0021260303  »  1)  .  „ ,98759115 

(0.9806724b) 


Reliability  =  0.988. 


RS  =  0.99626536  +  0.003174444 
R3  =  0.999439804 

Using  the  software  reliability  model, 

Ll  =  (0.999439804) (0.97) ( 1  -  0.98)  =  0.019389132 
L?  through  Ln  =  0 

Gi  =  (0. 999439804) (0 . 97 )  =  0.96945661 

G?  =  (0. 999439804) (0 . 97 ) ( 1  -  0 . 98 ) ( 1  -  0.89)  =  0.002132SC45 
G2  through  Gi(  =  0 
“  1  =  1 
4  2  =  1 

A3  through  A«  =  0 
A  =  1  -  0.019389132  =  0.98061087 

gives  Reliability  -  96945661  «  D  *  x  1)  .  0.99080017 

(0.98061087) 

Rel iabil ity  =  0.991. 

The  following  table  compares  the  reliability  values  that  are  obtained 
by  using  less  than  n  alternates  in  this  example. 


Number  of 
Alternates  Used 

Recovery  Block 
Reliability  Value 

Overall  Software 

Rel iabil ity  Value 

n  =  3 

0.98564 

0.977 

n  =  4 

0.99627 

0.988 

n  =  5 

0.99944 

0.991 

n  =  6 

0.99992 

0.991 

Table  11.  Accuracy  Effects  on  This 'Example  When  Less 
Than  n  Alternates  are  Actually  used 
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Discussion  of  the  Results: 

This  is  as  expected.  As  stated  in  Section  6.1.2,  by  actually  using 
fewer  than  the  n  alternates,  the  reliability  values  for  the  recovery  block 
and  the  overall  software  will  generally  decrease.  However,  as  was  seen  in 
the  case  with  n  =  5,  by  not  using  the  sixth  alternate  (which  has  a  reliability 
value  of  0.86  in  this  example),  an  extremely  slight  increase  in  reliability 
was  found. 

Example  3 

Figure  7  shows  a  possible  variation  of  a  forward  recovery  block. 

For  this  example,  the  number  of  alternates  will  be  three.  The  reliability 
value  for  each  of  the  software  components  is  listed  in  Table  12. 


Software  Component 

Reliability  Value 

Alternate  1 

0.80 

Alternate  2 

0.70 

Alternate  3 

0.90 

Acceptance  Test 

0.98 

Any  Process 

0.97 

Rollback 

0.95 

Rol 1 -Forward 

0.96 

Table  12.  Reliability  Values  for  the  Software  Components 
in  the  Figure  that  Represents  a  Variation  of 
the  Forward  Recovery  Block 


The  transfer  function  for  the  recovery  block  (RB)  is 

RB  =  G]  +  (1  -  G i ) G2  +  (1  -  G i ) ( 1  -  G2 ) G3 

=  0.80  +  (1  -  0.80)  x  (0.70)  +  (1  -  0.80)  x  (1  -  0.70)  x  (0.90) 
=  0.80  +  (0.20  x  0.70)  +  (0.20  x  0.30  x  0.90) 

=  0.80  +  0.14  +  0.054 


RB  =  0.994. 


The  variables  of  the  software  reliability  model  are 


Li  =  (0.994)  x  (0.98)  x  (1  -  0.95)  =  0.0*8706 
12  through  Ln  =  0 

Gi  =  (0.994)  x  (0.98)  x  (0.97)  =  0.9448S64 

G2  =  (0.994)  x  (0.98)  x  (1  -  0.95)  x  (1  -  0.96)  =  0.0019482 

G3  through  G«  =  0 

41  =  1 

A 

42  =  1 

43  through  4«  =  0 

4  ■  1  -  Lj  »  1  -  0.048706  =  0.951294 

Therefore, 

Reliability  =  JP.- 9448984  .*  +  (0-0019482  x  1)  _  g. 99532279 

(0.951294) 

Reliability  =  0.995. 

To  demonstrate  the  effect  on  accuracy  if  less  than  the  n  alternates 
(in  this  example  n  =  3)  are  actually  used,  the  reliability  of  the  recovery 
block  and  overall  software  reliability  value  will  be  re-calculated  for 
n  =  1  and  n  =  2. 


For  n  =  1, 

RB  =  0.80. 

With  the  software  reliability  model, 


Ll  =  (0.80)  x  (0.98)  x  (1  -  0.95)  =  0.0392 
12  through  Ln  =  0 

Gi  =  (0.80)  x  (0.98)  x  (0.97)  =  0.76048 

G2  =  (0.80)  x  (0.98)  x  (1  -  0.95)  x  (1  -  0.96) 

G3  through  G|<  =  0 

41  =  1 
43  =  1 

43  through  4«  =  0 
4  =  1  -  LX  =  1  -  0.0392  =  0.9608 


0.001568 


Reliability  =  (0-760*8  *  1)  ±  (0.001553  x  j) 

(0.9608) 


=  0.7931391 


Re  1 iabil ity  =  0.793. 

For  n  =  2, 

RB  =  0.80  +  0.14  =  0.94. 

With  the  software  reliability  model, 

Ll  =  (0.94)  x  (0.98)  x  (1  -  0.95)  =  0.04606 
L2  through  Ln  =  0 

Gi  =  (0.94)  x  (0.98)  x  (0.97)  =  0.893564 
G2  =  (0.94)  x  (0.98)  x  (1  -  0.95)  x  (1  -  0.96)  =  0.0018424 
G3  tnrough  Gk  =  0 
4!  =  1 

42  =  1 

43  through  4K  *  0 

4  =  1  -  Li  »  1  -  0.04606  «  0.95394 

Reliability  •  <0-893564  x  1)  ,+  (0.0018424  x  1)  ,  0.93864017 

(0.95394) 

Rel iabil ity  =  0.939. 

Table  13  compares  the  reliability  values  of  the  recovery  block 
and  the  overall  software  reliability  values  that  are  obtained  by  using  all 
or  less  than  the  n  alternates  in  this  example. 


Number  of 

Recovery  Block 

Overall  Software 

Alternates  Used 

Rel iabi 1 i ty  Value 

Rel iabi 1 i ty  Value 

n  =  1 

0.80 

0.793 

n  =  2 

0.94 

0.939 

n  =  3 

0.994 

0.995 

Table  13.  Comparison  of  Reliability  Values  When  Less 
Than  n  Alternates  are  Actually  Used 
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APPENDIX  III 

FEEDBACK  LOOP  CALCULATIONS 

For  basic  feedback  loops  such  as  those  shown  in  Figures  2,  4,  and 
5,  the  ideal  software  reliability  value  of  the  individual  blocks  and  overall 
software  reliability  is  1.0.  With  an  original  block  diagram  of  the  form 


with  y  =  +1  or  -1 


Figure  12.  Basic  Feedback  Loop 


the  block  diagram  transformation  to  eliminate  a  feedback  loop  gives  the  equiva¬ 
lent  block  diagram  of  the  form 


1  -  (y)  X  ( G ! G^H ) 


Figure  13.  Basic  Feedback  Loop  Equivalent 
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is: 


If  it  is  a  negative  feedback  loop,  the  equivalent  transfer  function 


g1g2 

1  +  G i G£  H 

With  Gj  =  1.0,  Gg  =  1.0,  and  H  =  1.0,  the  overall  reliability  value  would 
be  0.5,  which  is  undesirable.  It  is  desired  that  the  overall  software  reliability 
and  individual  block  reliability  values  be  1.0.  With  the  negative  feedback 
loop  and  these  goals,  there  are  tnree  cases  to  be  evaluated. 


Negative  Feedback  Loop, 


Want:  1  = 


Case  1: 

GlG2 


1  +  G1G2H 


If:  Gi  =  1.0  and  H  =  1.0 
G? 

Then:  =  1.0  or  1  +  G2  =  G2. 

1  +  G2 


Conclusion:  This  is  invalid. 


Negative  Feedback  Loop, 

Want:  1.0  = 


Case  2: 
GlG2 


1  +  G1G2H 


If:  G2  =  1.0  and  H  =  1.0 
Then:  1  +  G^  =  G^ 


Conclusion:  This  is  invalid. 


Negative  Feedback  Loop,  Case  3: 


Want:  1.0  = 


1  +  Gi G?H 


If:  G]_  =  1.0  ana  G2  =  1.0 

Then:  1  =1.0  cr  1  H  =  1 

1  +  h 


Conclusion:  H  =  0. 

If  it  is  a  positive  feedback  Iood,  the  eouivalent  transfer  function 


g1g2 

1  -  G}G£H 

As  G^  =>  1.0,  G2  s>  1.0,  and  H  =>  1.0,  the  overall  reliability  value  approaches 
+  «.  Again,  it  is  desired  that  the  overall  software  reliability  and  individual 
block  reliability  values  be  1.0.  There  are  three  cases  to  be  evaluated  with 
the  positive  feedback  loop. 

Positive  Feedback  Loop,  Case  1: 

GiG? 

Want:  1.0  =  1  f 

1  -  GjG2h 

If:  G[  =  1  C  ana  H  =  1  0 
g2 

Then:  =10  or  1  -  G?  =  G;  or  3  -  -•  C  b 

1  -  G2 

Conclusion:  This  is  undes i rab Je 


Positive  Feedback  Loop,  Case  2: 


Want:  1.0  = 


1  -  G^G/H 


If:  G2  =  1.0  and  H  =  1.0 


Then : 


1.0  or  1  -  Gi  =  G:  or  G  =  0. 


1  -  Gi 


Conclusion:  This  is  undesirable. 


Positive  Feedback  Loco,  Case  3: 


Want:  1.0 


1  -  GiG2H 


If:  Gi  =  1.0  and  G2  -  1.0 


=1.0  or  1  -  H  =  1 


1  -  H 


Conclusion:  H  =  0. 

By  comparing  the  results  of  the  positive  and  negative  feedback 
cases  (since  it  is  desired  that  the  software  reliability  model  should  accommodate 
both  of  these  options),  it  is  obvious  that  the  transfer  function  of  H  must 
equal  zero.  Therefore,  the  transfer  function  of  the  equivalent  block  in 
any  feedback  path  is  (1.0  -  reliability  value). 


APPENDIX  IV 

FEED-FORWARD  CALCULATIONS 


Figures  6  and  7  are  examples  of  block  diagrams  involving  feed-forwarc 
pa>.hs.  In  an  iaeai  situation,  the  reliability  value  of  the  individual  blocks 
(Gi,  G£,  G2,...Gn)  and  the  overall  software  reliability  (R)  are  1.0.  It 
is  important  to  remember  that  reliability  is  defined  such  that  0  <  Gi ,  G?, 

G3>...Gn,  R  <_  1.0.  Figure  14  shows  the  block  diagram  for  a  basic  feea-forwarc 
path. 
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Figure  14.  Basic  Feed-Forward  Path 


A  block  diagram  transformation  to  eliminate  the  feed-forward  loop 
gives  the  following  equivalent  block  diagram. 
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Figure  15.  Basic  Feed-Forward  Path  Equivalent 


XV 


Iceally,  if  the  reliability  value  of  the  individual  blocks  is  1.0, 
:hen  the  overall  reliability  should  be  1.0.  Therefore, 

Want:  1.0  =  G^  +  G? 

If:  Gj,  =  1.0 

Then:  G2  =  0  or  1.0  =  1.0  +  (1.0  -  G2) 

Conclusion:  The  transfe*-  function  of  the  ecuivalent  block  in  any 
feed-forward  path  is  (1.0  -  reliaoility  value). 


B 


/.  v.  ■  -  r.v.v. 


APPENDIX  V 

ANALYSIS  OF  SCOTT'S  RECOVERY  BLOCK  RELIABILITY  MODEL 


In  Scott's  recovery  block  reliability  model,  the  variables  are 
Ge'ined  as: 

P(C-j)  =  the  probaoility  of  alternate  i  executing  correctly; 

PCi)  =  1  -  P ( C i ) ; 

P(Cr)  =  the  probability  of  the  reccvery  program  executing  correctly; 

PC R)  =  1  -  P(Cr); 

P ( A t )  =  the  probaoility  of  accepting  an  incorrect  result; 

P(R:)  =  the  probability  of  rejecting  an  incorrect  result 

=  1  -  P(AX); 

P ( Rc )  =  the  probability  of  rejecting  a  correct  result; 

P ( Aq )  =  the  probability  of  accepting  a  correct  result 

=  l  -  P(RC); 

Type  1  Error  =  the  program  alternate  produces  an  incorrect  result, 

but  the  acceptance  test  labels  the  result  as  correct; 

Type  2  Error  =  the  final  alternate  produces  correct  results,  but 
the  acceptance  test  erroneously  determines  that 
the  results  are  incorrect; 

Type  3  Error  =  the  recovery  program  cannot  successfully  recover 

the  input  state  of  the  previous  alternate  in  prepa¬ 
ration  for  executing  another  alternate  or  could 
not  successfully  invoke  the  next  alternate; 

Type  A  Error  =  the  last  alternate  produces  incorrect  results  and 
the  acceptance  test  judges  that  the  results  are 
i ncorrect ; 

n  =  the  number  of  alternates;  and 

R  =  the  reliability. 


The  results  of  Scott's  recovery  block  reliability  model  will  be 
compared  to  those  that  would  be  obtained  in  the  software  reliability  model 


Iki 


Figure  17.  Special  Case  Recovery  Block  with  Only  One  Alternate 


with  an  overall  transfer  function  of  R  =  Gj  x  G2.  This  special  case  with 
n  =  1  is  computed  with  Scott's  recovery  block  reliability  model  as 


R  =  1  -  [Type  1  Error  +  Type  2  Error  +  Type  3  Error  + 
Type  4  Error], 

with  Type  1  Error  =  P(Ii)P(Aj); 

Type  2  Error  =  P(Cj)P(Rc); 

Type  3  Error  =  0;  and 
Type  4  Error  =  P( I i )P( Rj ) - 


By  substitution. 


GiG2  =  1  -  [P( I !  )P(AI )  +  P ( C ! ) P ( RC )  +  P( I i)P(Ri )] • 


By  Torthe-"  Substitution, 


GiG2  =  1  -  { [1  -  P(Ci)][l  -  P{ R;  )I  -  P(Ci)[l-?(Ac)j  + 
[1  -  P(C1)]P(RI);. 

Multiplying  out  the  factors  gives 

G’G?  =  1  -  >1  +  P(Ci)P(Rr)  -  P(Ci)  -  P(R-)  -  P{Ci) 

-  P(Ci  )P(AC)  +  P(R-)  -  P ! C T. ) P ( R r ) }  . 

This  can  be  recuceO  to 

GiG2  =  1  -  [1  -  P ( C x ) P ( Ac ) ]  =  P(Ci)?(Ac) . 

This  is  as  expected,  with  Gi  =  P ( Ci )  and  G2  =  P(Aq). 

For  n  =2, 


1  -  g^g2h 


=  1  -  [Type  1  Error  +  Type  2  Error  + 
Type  3  Error  +  Type  4  Error], 


By  substitution. 


1  -  G^G2H 


[ { P ( 1 1 ) P ( A ! )  +  [P( I i )P(Aj )  -  0]  x 
[  P(CR)P(I2)  ]  x  [P(C]_)P( Rc )  + 

P(Ii) 

p( 1 1  )P(Rl )]1  +  { P(C1)P(Rc)P(Cr)P(C2)  X 

[P ( Rc )  +  P ( I i ) P( Rj )  ]i  + 

P  ( C ! ) 

|P(Ci)P(RC)P(Ir)  +  P( Ii)P(RI)P(Ir)1  + 
!P(Ii)P(Ri)P(Cr)P(I2)[P(RI)  + 

P ( C i ) P ( Rc )  ]j]. 


Multiplying  out  the  factors  and  cancelling  alike  numerator  anc  aenominator 
terms  gives 


=  1  -  [P( 1 1 ) P ( A j )  +  P(AI)P(CR)P(I2)P(C1)P(RC)  + 

1  -  GiG2H  P(Aj )P(Cr)P(  I2)PCi)P(Ri )  + 

P(Ci)P(Rc)P(CR)P(C2)P(RC)  + 
P(RC)P(Cr)P(C2)P(Ii)P(Ri)  + 

P(Ci)P(RC)P(IR)  +  ?(Ii)P(R:)P;Ir)  + 
P(Ii)P(Ri)P(Cr)P{I2)P(Ri)  + 
P(R:)?(CR)P(I2)P{C1)P(Rc)]. 

By  substitution, 

G1G2  =  1  -  { [1  -  P(Ci)][l  -  P(Ri )]  + 

1  -  G^H  [1  -  P(Ri)]P(Cr)[1  -  P(C2)]P(C!)  x 

Cl  -  P(AC)]  +  [1  -  P(Rl)]P(CR)  x 
[1  -  P(C2)][1  -  P(C1)]P(RI )  + 

P(Ci)[l  -  P(AC)]P(Cr)P(C2)[1  -  P ( Ac ) ]  + 

[1  -  P(AC ) ]P(CR)P (C2 ) [ 1  -  P(C1)]P(RI )  + 
P(Ci)[l  -  P(AC)][1  -  P(CR)]  + 

[1  -  P(Ci)]P(Ri)[l  -  P(CR)]  + 

[1  -  P(Ci)]P(Ri)P(Cr)[1  -  P(C2)]P(Ri)  + 
P(Rl)P(CR) [1  -  P(C2)]P(C!)[1  -  P ( A c ) ] }  • 

With  the  factors  multiplied  out, 

G1G2  =  1  -  [1  +  P(C1)P(RI)  -  P(CX)  -  P( Rj )  +  P(C1)P(Cr) 

1  -  GiG2H  P(Ci)P(C2)P(Cr)  -  P(Ci)P(Cr)P(Ri)  + 

P(C1)P(C2)P(Cr)P(Ri)  -  P(AC)P(C1)P(CR)  + 
P(Ac)P(CiJP(C2)P(Cr)  +  P(AC)P(C1)P(CR)P(RI)  - 
P(AC)P(Ci)P(C2)P(Cr)P(RI)  +  P(Cr)P( R: )  - 
P(C1)P(Cr)P(RI)  -  P(C2)P(Cr)P(Ri)  + 
P(C1)P(C2)P(Cr)P(RI)  -  P(Cr)P(Ri)P(Ri)  + 
P(C1)P(Cr)P(RI)P(Ri)  +  P( C2 )P( Cr ) P ( Rj ) P ( Rj )  - 
P(Ci)P(C2)P(Cr)P(Ri)P(Ri)  +  P(Ci)P(C2)P(CR)  - 


iLSjM. 


?{Ac)PtC1)P(C2)P(Ca)  -  P(Ac)P(C1)P(C2)P(Cr)  - 
PvAc)P(Ac)P(C1)P(C2)?(C^  t  P(C2)P(Cr)P(R:)  - 
P(C^P(C2)P(Cr)P(R:)  -  P(Ac)P(C2)P(Cr)P{Rt)  + 
D(AC)P(C1)P(C2)P(CR)P{.R:)  +  P(CL)  -  P(C:)P(CR) 
P(Ac)P(Ci)  +  P(A C)?(C1)P(CR)  +  P ( R ! )  - 
P(Cr)P(Rj )  -  P(CX )P(R: )  -  P(C1)P(CR)P(Rt)  + 
P(Cr)P(Ri)P(RI)  -  P(C2)P(Cr)P(P;)P(R;)  - 
P(Ci)P(Cr)P(Rt)P(R:)  + 

p(Ci)p(c2)p(cr)p(R:)p(r:)  +  p(c1)p(cr)p(ri)  - 

P(AC)P(Ci)P(Cr)P(Rt)  -  P(C1)P(C2)P(Cr)P(Rt)  + 
P(Ac)P(Ci)P(C2)P(CR)P(R:)j. 


By  cancellation, 


G1G2 

1  -  G^G2H 


[1  -  P(Ac)P(Ci)P(C2)P(CR)  + 
P(AC)P(AC)P(C1)P(C2)P(CR)  - 

P(AC)P(C2)P(CR)P(RI)  -  P(AC)P(C i)  + 
P(AC)P(Ci)P(C2)P(Cr)P(RI)]. 


Rearranging  gives 


G1G2  «  P(AC)[P(Ci)P(C2)P(Cr)  -  P(AC)P(Ci)P(C2)P(Cr)  + 
1  -  G].G2H  P(C2)P(Cr)P(R! )  +  P ( C i )  - 
P(Ci)P(C2)P(CR)P(RI)]. 


For  n  =  3, 


1  -  GxG2H 


1  -  [Type  1  Error  +  Type  2  Error  + 
Type  3  Error  +  Type  4  Error], 


By  substitution, 


=  1  -  [|  P(  1 1 ) P ( A  j )  4 

1  -  G!G2H  P(AI)P(CR)P(I2)[P(C1)P(RC)  +  P(Ii)P(Rj)]  + 

P(Ai)P(Cr)P(I2)[P(C1)P{Rc)  4  P( I1)P(RI )]  x 


C  p(Cr)p(i3}  ]  [  P  ( C  ? )  P  ( R,- )  +  p(  l2)p(  RI )];  + 
P(I2) 

{ P(C1)P(C2)P(CR)P(RC)P( RC)P(Cr)P(C3)  x 
[?(RC)  +  p(I2)p(Rl)  ]  +  P(C2)P(Cr)P(Ii)  x 
P(C2) 

P(Rc)p(Rl)P(CR)P(C3)CP(RC)  +  p(J2)p(Rl)  ]i  + 

P(C2) 

{ P(Ci)P(Rc)P(Ir)  +  P(Ii)P(Ri)P(Ir)  + 
[?(C1)P(RC)P{IR)  +  P(Ii)P(Rt)P(Ir)]  x 

p(Cr)[?(c2)p(rc)  +  P( I3)P( R: )]}  + 

{[P(Ii)P(Ri)P(Cr)P(I2)[P(Ri)  +  p ( c 1 ) P ( RC )  ]  x 

P(I  l) 

P(Cr)P(I3)[P(Ri)  +  P(C2)P(rc)  ]}]. 

P(I2) 

By  multiplying  out  the  terms  (and  arranging  the  coefficients  in  alphabetical  and 
numerical  order), 

GlG2  =  1  -  {P(Ai)P(Ii)  +  P(AI)P(C1)P(CR)P(I2)P(RC)  + 

1  -  GiG2K  P(Ai)P(Cr)P(I1)P(I2)P(Ri)  + 

[P(Ai)P(C1)P(Cr)P(Cr)P(I3)P(RC)  + 
P(Ai)P(Cr)P(Cr)P(I1)P(I3)P(Ri)]  x 
[P(C2)P( RC)  +  P(I2)P(Ri)]  + 
P(Ci)P(C2)P(C3)P(CR)P(CR)P(RC)P(Rc)P(rc)  + 
P(Ci)P(C3)P(CR)P(CR)P(I2)P(RC)P(Rc)P(ri)  + 
P(C2)P(C3)P(Cr)P(CR)P(I1)P(RC)P(RC)P(Ri)  + 
P(C3)P(Cr)P(Cr)P(I1)P(I2)P(RC)P(Ri)P(RI)  + 
P(Ci)P(Ir)P(RC)  +  P(Ii)P(Ir)P(Ri)  + 
P(Ci)P(C2)P(CR)P(IR)P(Rc)P(RC)  + 
P(Ci)P(Cr)P(I3)P(Ir)P(Rc)P(RI)  + 
P(C2)P(Cr)P(I1)P(Ir)P(Rc)P(RI)  + 
P(Cr)P(I1)P(I3)P(Ir)P(RI)P(RI)  + 
[P(Cr)P(Cr)P(Ii)P(I2)P(I3)P(Ri)P(Ri)  + 
P(Ci)P(Cr)P(Cr)P(I2)P(I3)P(Rc)P(Ri)]  x 
[P(Rl)  +  P(C2)P(RC)  ])  . 

P(I2) 


By  substitution  [to  eliminate  the  P(Aj)  and  P ( I )  terns], 

r 

GlG2  =  1  -  i[l-?(Ci)][l  -  P( Rr ) ]  + 

1  -  GiG2H  [1  -  P(Ri)]P(Ci)P(CR)][1  -  P(C2)][1  -  P(Ac)j  - 

[1  -  P(Ri) ]P(CR)[1  -  P(Ci)][l  -  P(C2)]P(Ri)  + 

{[i  -  p(R:)][p(Ci)p(cr)p(cr)[i  -  p(c3)][i  -  p(ac)]  * 

[1  -  P(Rj)]P(Cr)P(CR)[1  -  P(Ci)][l  -  P(C3) Jp( R: )}  x 
1  P(C2)[1  -  P ( Ac ) ]  +  [1  -  P(C2)]P(Rj);  + 
P(C1)P(C2)P(C3)P(CR)P(CR)[1  -  P ( Ac ) ] [ 1  -  P(AC)J[1  -  P(AC  )]  , 
P(C1)P(C3)P(CR)P(CR)[1  -  P(C2)][I  -  P ( Ac ) j [ 1  -  ?(Ar ) ]P( R* )  - 
P(C2)P(C3)P(CR)P(CR)[1  -  P(Ci)][l  -  P(AC)][1  -  P (AC ) ]  x 
p ( R I )  +  P(C3)P(CR)P(CR)[1  -  P(Ci)][l  -  P(C2)][1  -  P(AC)3  X 
P( Rr )P (Rt )  +  P ( C ! ) [ 1  -  P(CR)][1  -  P ( Ac ) ]  +  [1  -  P ( C i ) 3  X 
[1  -  P(CR)]P(RX )  +  P(C!)P(C2)P(CR)[1  -  P(CR)][1  -  P(AC)]  x 
[1  -  P ( Ac ) ]  +  P(C1)P(CR)[1  -  P(C3)][1  -  P(CR)][1  -  P(AC)]  x 
P(Rl)  +  P(C2)P(Cr)[1  -  P(Ci)][l  -  P(CR)][1  -  P ( Ac ) ]  x 
P(Rl)  +  P(Cr)[1  -  P(C])1[1  -  P(C3)][1  -  P(CR)]P( R] )  x 
P(Rl)  +  P(CR)P(CR)[1  -  P(C!)][1  -  P(C2)][1  -  P(C3)]  x 
P(Rl)P(Rl)  +  P(Ci)P(CR)P(CR)[l  -  P(C2)][1  -  P(C3)]  x 
[1  -  P(Ac)]P(Rl )  x  {  P ( R T )  +  P(C2)[1  -  pCAc) 3 ; ^ 

[1  -  P(C2)]  j 

Multiplying  the  terms  out  gives: 

GlGz  =  1  -  [1  -  P ( C i )  -  P(RX)  +  P(C1)P(RI)  +  P(Ci)P(CR)  - 

1  -  GXG2H  P(C!)P(C2)P(CR)  -  P(Ac)P(Ci)P(CR)  +  P(AC)P(C!)P(C2)P(CR)  - 

P(Ci)P(Cr)P(Ri)  +  P(C!)P(C2)P(CR)P(Ri)  +  P(AC)P(C1)P(CR)P(R;)  - 
P(AC)P(Ci)P(C2)P(CR)P(R i)  +  P(Cr)P(Ri)  -  P(C1)P(CR)P(RI)  - 
P(C2)P(Cr)P(Ri)  +  P(Ci)P(C2)P(Cr)P(Ri )  - 

P(CR)P(R1)P(RI)  +  P(C1)P(CR)P(RI)P(Ri)  +  P(C2)P(CR)P(R1)P{Ri)  - 
P(C1)P(C2)P(CR)P(RI)P(RI)  +  P(C1)P(C2)P(CR)P(CR)  - 
P(C!)P(C2)P(C3)P(CR)P(CR)  -  P(AC)P(Ci)P(C2)P(CR)P(CR)  + 
P(AC)P(C1)P(C2)P(C3)P(CR)P(CR)  -  P(C1)P(C2)P(Cr)P(Cr)P(Ri )  - 
P(C1)P(C2)P(C3)P(CR)P(CR)P(R i)  +  P(AC)P(C1)P(C2)P(Cr)P(Cr)P(«i ) 


P ( AC ) P ( C i ) P ( C2 ) P v c 2 } P c CR ) P ( Cr ) P ( R ; )  ♦  P(C2)P(Cr)P(Cr)P(R:)  - 
P(Ci)P(C2)P(CR)PlCR)P(R;)  -  P'C2}?(C3)P{Cr)P(Cr)P{Rt)  - 
P(C1)P(C2)?(C3)P(CR)PICr)P(R:)  -  P(C2)P(Cr)P(Cr)P(R: ) P ( R ; )  - 

p  ( c  i )  p  ( c  2 )  p  ( c  R )  f  ( c  R )  p  (  r  : )  p  ( ?. : )  +  p  ( c  2 )  p  ( c  2 )  p  ( c  R )  f  { c  r  )  p  (  r  : )  p  {  r  : ; 

P(C1)P(C2)P(C3)P(CR;P(CR)?(R:)P(R:)  -  P(Ac)P(Ci)P(C2)P(Cr)P'C=. 
P(AC)P(Ci)P(C2)P(C3)P (Cr)P(Cr)  +  P ( AC ) P ( Ac ) P ( C i ) P ( C  2 ) P ( C  R ) P ( C  R 
P(AC)P(AC)P(C1)P(C2)P(C3)P(CR}?(CR)  - 
p(Ac)P(C:)P(c2)p(cr)p(Cr)?(r:)  - 
P(Ac)P(C;)P(C2)P(C3)P(CR)P(Cr)P(R:)  - 
P ( Ar ) P ( Ar } P ( C  3 ) P ( C £ ) P ( C R ) ? ( C R ; ? ;  R  * )  - 
P(Ac)P(Ac)P(Ci )P(C2)P(C3)P(CR)?  :CR)P'  Rr )  - 
P(AC)P(C2)P(Cr)P(CR)P(Rt)  + 

P(Ac)P(Ci)P(C2)P(CR)P(CR)P(RI )  + 

P(AC)P(C2)P(C3)P(CR)P(CR)P(RI)  - 
P(AC)P(C1)P(C2)P(C3)P(CR)P(CR)P(RI)  + 
P(AC)P(C2)P(CR)P(CR)P(RI)P(RI)  - 
P(AC)P(C1)P(C2)P(Cr)P(Cr)P(RI)P(RI)  - 
P(AC)P(C2)P(C3)P(CR)P(CR)P(RI)P(Ri)  + 
P(AC)P(C1)P(C2)P(C3)P(Cr)P(CR)P(Ri)P(RI)  + 

P(C1)P(Cr)P(Cr)P(R: )  -  P(C1)P(C3)P(Cr)P(Cr)P(RI)  - 
P(AC)P(C1)P(CR)P(CR)P(RI)  +  P(Ac)P(Ci)P(C3)P(Cr)P(Cr)P(RI)  - 
P(C1)P(Cr)P(Cr)P(RI)P(Ri)  +  P(Ci)P(C3)P(Cr)P(Cr)P(Ri)P(Ri)  + 
P(AC)P(Ci)P(Cr)P(Cr)P(RI)P(RI)  - 
P(AC)P(C1)P(C3)P(Cr)P(Cr)P(Ri)P(RI)  + 

P(Cr)P(Cr)P(RI)P(Ri)  -  P(C1)P(Cr)P(Cr)P(Ri)P(Ri)  - 
P(C3)P(Cr)P(Cr)P(RI)P(RI)  +  P(C1)P(C3)P(Cr)P(Cr)P(RI)P(R:)  - 
P(Cr)P(Cr)P(Ri)P(R:)P(Ri)  + 

P(C1)P(Cr)P(Cr)P(R:)P(RI)P(Rt)  +  P(C3)P(CR)P(CR)P(Rt)P(RI)P(Rt) 
P(C1)P(C3)P(Cr)P(Cr)P(RI)P(Ri)P(R:)  -  P(Ci)P(C2)P(Cr)P(Cr)P(RI ) 
P(C1)P(C2)P(C3)P(Cr)P(Cr)P(Ri)  +  P(Ac)P(C1)P(C2)P(Cr)P(Cr)P(Rt) 
P(AC)P(C1)P(C2)P(C3)P(Cr)P(Cr)P(Ri)  + 
P(C1)P(C2)P(Cr)P(Cr)P(RI)P(Ri)  - 
P(C1)P(C2)P(C3)P(CR}P(CR)P(Ri)P(RI)  - 
P(AC)P(C1)P(C2)P(Cr)P(Cr)P(Ri)P(RI)  + 
P(AC)P(C1)P(C2)P(C3)P(CR)P(CR)P(RI)P(Rt)  - 
P(C2)P(CR)P(Cr)P(R:)P(RI)  +  P(Ci)P(C2)P(CR)P(CR)P(RI)P(Rj)  ♦ 
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p(c2>p(c3)p(Cr)p(c r)p(R;)p(R:)  - 

P(Ci)P(C2)P(C3)P(CR)P(CR)P(R;)P(R;)  + 

p(c2)p(cr)p(Cr)p'.r: ) p ( r : ) p ( r i )  - 

P(Ci)P(C2)P(CR)P(CpJP(R:)P(R:)P:s:)  - 
P(C2)P(C3)P(Cr)P(CR)P(RI)P(RI)P(Rt)  - 
P(C1)P(C2)P(C3)P(Cr)P(CR)P(R:)P(R;)?'R;)  - 
P(C1)P(C2)P(C3)P(CR)P(CR)  - 
P(PC ;c(Ci)P(C2;P(C3)P(CR)P(CR)  - 
p  ( R  C  ) 0  c c  i ) p  ( c  2  ^ ?  v c  2 ) p  ( c  R ) p '  C  R )  - 

p(Pc)p'C1)P(C2)P(C3)P(CR)P(CR)  * 

p(ac;p(Ac)p(Ci)p(c2}p(c3)p(Cr)p(cr)  + 

P(AC)P(Ac)P(Ci)P{C2)P(C3)P(CR)P(CR)  + 
P(AC)P(AC)P(C1)P(C2)P(C3)P(CR)P(CR)  - 
P(AC)P(AC)P(AC)P(C1)P(C2)P(C3)P(CR)P(CR)  + 
P(C1)P(C3)P(CR)P(CR)P(RI)  -  P(AC)P(C1)P(C3)P(CR)P(CR)P(RI)  - 
P(AC)P(C1)P(C3)P(CR)P(CR)P(Ri)  + 
P(AC)P(Ac)P(C1)P(C3)P(CR)P(CR)P(Ri)  - 
P(C1)P(C2)P(C3)P(CR)P(CR)P(RI)  + 
P(Ac)P(C1)P(C2)P(C3)P(Cr)P(CR)P(Ri)  + 
P(AC)P(C1)P(C2)P(C3)P(CR)P(CR)P(Ri)  - 
P(AC)P(AC)P(C1)P(C2)P(C3)P(CR)P(CR)P(RI)  + 
P(C2)P(C3)P(CR)P(CR)P(Ri)  -  P(AC)P(C2)P(C3)P(Cr)P(CR)P(RI)  - 
P(AC)P(C2)P(C3)P(Cr)P(Cr)P(RI)  + 
P(AC)P(AC)P(C2)P(C3)P(CR)P(CR)P(RI)  - 
P(C1)P(C2)P(C3)P(Cr)P(Cr)P(Ri)  + 
P(Ac)P(C1)P(C2)P(C3)P(Cr)P(Cr)P(RI)  + 
P(AC)P(C1)P(C2)P(C3)P(Cr)P(Cr)P(Rj)  - 
P(AC)P(AC)P(Ci)P(C2)P(C3)P(Cr)P(CR)P'R;)  ♦ 
P(C3)P(CR)P(CR)P(RI)P(RI)  - 

P(AC)P(C3)P(CR)P(CR)P(RI)P(RI)  -  P(C2)P(C3)PtCR)P(CR)P(R]  ) P ( R j  )  • 
P(Ac)P(C2)P(C3)P(Cr)P(Cr)P(Ri)P(R:)  - 
P(C1)P(C3)P(Cr)P(Cr)P(RI)P(RI)  + 
P(AC)P(C1)P(C3)P(Cr)P(Cr)P(Ri)P(Ri)  ♦ 
P(C1)P(C2)P(C3)P(Cr)P(Cr)P(RI)P(R1)  - 


P(Ac)P(C1)P(C2)P(C3)P(Cs)P(CR)?(R:)F(Rt)  -  P(Ci)  - 
P(AC)P;C!)  -  P { C ! ) P ( C R )  -  P(Ac)P(Ci)P(CR)  -  P(R;)  - 
P(CR)P(RZ )  -  P(Ci )P(Rr )  +  P(Ci)P(CR)P(RI )  +  F(Ci)P(C2)P(Cr)  - 
P(Ci)P(C2)P(CR)P(CR)  -  P(AC)P(C1)P(C2)P(CR)  - 
P(AC)P(C1)P(C2)P(CR)  -  P(AC)P(C1)P(C2)P(CR)P(CR)  - 
P(AC)P(C1)P(C2)P(CR)P(CR)  -  P(Ac)P(Ac)P(Ci)P(C2)P(CR)P(CR)  + 
P(C1)P(Cr)P(RI)  -  P(C1)P(C3)P(Cr)P(RI)  -  P(AC)P(C:)P(CR)P(R:)  - 
P(AC)P(Ci)P(C3)P(CR)P(R:)  -  P(Ci)P(Cr)P(Cr)P(R:)  * 
P{Ci)P(C3)P(CR)P(CR)P(R:)  +  P ( AC ) P ( C ! ) P ( C R ) P ( C R ) ? i R : )  - 
P(Ar)P{Cz)P(C3)P(CR)P(CR)P(R*)  + 

P ( C 2 ) ? ( C R ) P ( R : )  -  P(Ci)P(C2)P(CR)P(Rr)  - 
P(AC)P(C2)P(Cr)P(Rt)  +  P(Ac)P(Ci)P(C2)P(CR)P(RI)  - 
P(C2)P(CR)P(CR)P(RI)  r  P(Ci)P(C2)P(CR)P(CR)P(RI)  - 
?(Ac)P(C2)P(Cr)P(Cr)P(RI)  -  P(Ac)P(C1)P(C2)P(Cr)P(Cr)P(Ri)  + 
P(Cr)P(Ri )P(Rj )  -  P(C1)P(CR)P(Rt)P(RI)  - 
P(C3)P(Cr)P(RI)P(RI)  +  P(C1)P(C3)P(CR)P(RI)P(RI)  - 
P(CR)P(CR)P(RI)P(RI)  +  P(Ci)P(Cr)P(Cr)P(RI)P(RI)  + 
P(C3)P(Cr)P(Cr)P(RI)P(RI)  -  P(C1)P(C3)P(Cr)P(Cr)P(RI)P(Ri)  + 
P(Cr)P(Cr)P(RI)P(RI)P(RI)  -  P(C1)P(Cr)P(Cr)P(Ri)P(RI)P(Ri)  - 
P(C2)P(Cr)P(Cr)P(RI)P(RI)P(RI)  + 
P(C1)P(C2)P(Cr)P(Cr)P(Ri)P(RI)P(RI)  - 
P(C3)P(Cr)P(Cr)P(RI)P(Ri)P(RI )  + 
P(C1)P(C3)P(Cr)P(Cr)P(Ri)P(RI)P(RI)  + 
P(C2)P(C3)P(Cr)P(Cr)P(Ri)P(RI)P(RI)  - 
P(C1)P(C2)P(C3)P(Cr)P(Cr)P(Ri)P(RI)P(RI)  + 
P(C1)P(Cr)P(Cr)P(Ri)P(Ri)  -  P(C1)P(C2)P(Cr)P(Cr)P(RI)P(Ri)  - 
P(AC)P(Ci)P(Cr)P(Cr)P(RI)P(RI)  + 
P(AC)P(C1)P(C2)P(Cr)P(Cr)P(RI)P(RI)  - 
P(C1)P(C3)P(Cr)P(Cr)P(Ri)P(Rj)  + 
P(Ci)P(C2)P(C3)P(CR)P(CR)P(RI)P(RI)  + 
P(AC)P(C1)P(C3)P(Cr)P(Cr)P(RI)P(Ri)  - 
P(AC)P(C1)P(C2)P(C3)P(Cr)P(Cr)P(RI)P(RI)  + 
P(C2)P(Cr)P(Cr)P(RI)P(RI)  -  P(Ac)P(C2)P(Cr)P(Cr)P(Ri)P(ri) 
P(Ci)P(C2)P(Cr)P(Cr)P(Ri)P(Ri)  + 


‘  /Tf  .'nVA  ",  V  V  V  ■ 


By  cancellation, 


1  *  G3G2H 


p(a.c)p(c1)p(c2)?(cr)p(c3)?(R:)p(r:;i  - 
P1C2)P(C3)?(CR)?(C3)P(R:)P(R;)  - 
P(Ac)P(C2)P(C3)P(CR)P(Ca)P(R:)P(R:)* 
PtC:.lP(C2lP(C3)P(CR)P;CR)P{R:)P(R:)  - 
F  ( Ac )  ?  ( C  3 )  ?  ( C  2 )  P  ( C  3 )  P  ( C  R  ]  ?  ( C  R )  P  ( R  * )  ?  ( R ; }  + 
p ( c  i ) p ( c  2 ) p ( c R ) ? ( c R ) p ( r : )  -  p ( ac  )  p ( c , ) ? : c  2  • p ; 
p(Ac)pic:)p(c2)p(cr)P(cr)p(r:)  - 
P  ( AC }  ?  C  AC ) P  ( C  -.  ) P  ( C 2 ) P  ( C r )  ?  ( C R ) P  ( R ; )  - 
p:c1)P(c2jP(c3)p(cR:'?(cR)p(R:)  + 

P  ( A  c  >  P  t  c : )  P  ( c  2 )  p  •:  C  3 )  P  ( c  R )  P  ( c  s )  P 1  r  : )  + 
P(Ac)P(Ci;P(C2)?(C3)?(CR)P;CR)P(R:)  •• 

P 1 Aq ) ? ( Ac ) P ( C 1 ) P ( C  2 ) P ( C  3 ) P ( C  R ) P ( C  R ) P 1 R : . 


*->'?'  3  ■ 


[1  +  P(C1)P(C2)P(CR)P(Ri )  +  P(C2)P(CR)P(RI)P(RI)  - 
P(Ci)P(C2)P(CR)P(RI)P(RI)  -  P(Ci)P(C2)P(Cr)P(Cr)P(R:)  * 
P(AC)P(C1)P(C2)P(CR)P(Cr)P(RI)  +  P(C1)P(C2)P(Cr)P(Cr)P(Rt  )P( R; ) 
P(C2)P(Cr)P(Cr)P(Rt)P(RI)  -  P(AC)P(Ci)P(C2)P(C3)P(Cr)P(Cr)  + 

2  x  P(Ac)P(Ac)P(C1)P(C2)P(C3)P(Cr)P(Cr)  - 
P(Ac)P(AC)P(Ac)P(C1)P(C2)P(C3)P(Cr)P(Cr)  + 

P(Ci)P(C3)P(Cr)P(Cr)P(R])  -  2  x  P(AC)P(C1)P(C3)P(Cr)P(Cr)P(R: )  ■ 
P(AC)P(Ac)P(C1)P(C3)P(Cr)P(Cr)P(RI)  + 

2  x  P(Ac)P(C1)P(C2)P(C3)P(Cr)P(Cr)P(Ri)  - 
2  x  P(Ac)P(Ac)P{C1)P(C2)P(C3)P(Cr)P(Cr)P(Ri)  - 
P(Ac)P(C2)P(C3)P(Cr)P(Cr)P(RI)  + 
P(Ac)P(Ac)P(C2)P(C3)P(Cr)P(CR)P(Ri)  - 
P(Ac)P(C3)P(Cr)P(Cr)P(RI)P(RI)  -  P(Ac)P(Ci)  - 
P(C1)P(C3)P(Cr)P(Rt)  +  P(AC)P(C1)P(C3)P(Cr)P(Rt)  - 
P(AC)P(C2)P(Cr)P(RI)  -  P(C3)P(Cr)P(RI)P(Ri)  + 
P(C1)P(C3)P(Cr)P(Ri)P(Ri)  +  P(C3)P(Cr)P(Cr)P(Rt)P(Ri)  - 
P(C1)P(C3)P(Cr)P(Cr)P(RI)P(R])  + 
P(AC)P(C1)P(C3)P(Cr)P(Cr)P(RI)P(Ri)  + 
P(AC)P(C2)P(C3)P(Cr)P(Cr)P(R1)P(Ri)  - 
P(AC)P(C1)P(C2)P(C3)P(Cr)P(Cr)P(RI)P(Ri)]. 


L  -  r  v- ;  ,  r  l  : 


1  -  r  G  •  r  ; ; 


- '  ,  r ;  u  ;  i  ■"  ■■  -  *  -  r  .  -  ?  '  r  . u  p  . 


u  *•  "  P>r  __1  -  P  '  Ar  )  .  P  '  Ar  '■  P  ' 

'i  -  PiPc1.  x  • ?  v  c  i )  I p  ;  c  ? )  * 

P <  Ac ; ? ;3  •  :  =  ;C:)  *  ? '  C  r  1  -  2: 

[i  -  ;  x  •  p ( c 2 )  -  p ' c 2 : 

?  ( c  q )  ? ,  c ;  p  ,  r  : ;  p  :  p  : ) . 


Scot:  s  ce*inticn$  of  t~e  vagaries,  it  would  oe  expected  tra: 


a.  Tne  recovery  block,  G^,  with  its  n  a'tsT.ates  be  a  '"unction 
of  P '  C i' ) ; 

b.  Toe  acceptance  test,  G?,  be  a  function  o*  ?,A r]  anc.C'- 
P ( R I ) ;  and 

c.  The  rollbacK,  H,  be  a  function  of  P(Cq). 


Although  this  was  true  for  the  special  case  with  n  =  1,  the  cases  with  n  =  2 
and  n  =  3  show  that  the  two  models  are  too  distinct  in  their  methocs  to  be 
compared . 
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APPENDIX  VI 

ANALYSIS  OF  SCOTT'S  N-VERSION  PROGRAMMING  RELIABILITY  MODEL 


.n 


a^e  ce' • re:  as : 


:tt  s  .i-ve^sion  c 3 c ^ a mrr. :  rc  relief. 


"cce  ,  tre  va  ■ 


P;C’)  -  tie  croDaD’  I :  ty  or  version  i  executing  correctly; 

P  \  I  ■. )  =  tre  probability  c*  version  i  executing  '  ncor-ec:  ’  y , 

1  -  P*C- ) ; 

Type  !  --’•or  5  a''  cuffs  disagree; 

Z  £--:r  =  ar,  inco-ftt  output  ccc-rs  -ere  far  cr:e; 

Type  3  Error  =  thef  is  ar  error  :r  tre  vct'nc  (cecsior  al: 
procedure , 

n  =  tre  nur,oer  of  verS'ons;  and 
R  1  tre  re'  i  a D : I i t y  . 


The  resu’ts  of  Scott's  N-version  programming  reliability  model 
w1 1 !  be  compared  to  those  that  would  be  obtained  in  the  software  reliability 
mode!  that  has  beer  proposed.  Tre  block  diagram  for  the  N-version  software 
tnat  is  described  by  Scott  would  look  like. 


P 

U 

T 


Version  1 


v  e  r  s i on 


Version  3 


Version  n 


Deci sion 
A!  con  tnn 
( Votef 


C 

U 

P 

U 

T 


Figure  18.  Basic  N-Yersion  Software 


For  the  proposec  softwa-e  rel  iaoi  l  ’  C..  mcce' ,  f  is  cessed  tra 
the  individual  reliability  values  for  tre  n  versions  be  combine:  to  form 
one  reliability  value  (or  transfer  f-nefon!  for  tre  group.  Tre  block  c 
for  this  might  be 


Deci sion 
A1 cori thm 
(Voter) 

(G2) 


Figure  19.  Basic  N-Version  Software  Equivalent 


wit"  an  overall  trans*er  function  of  R  =  G;  x  G2. 

A  simple  case  to  exam'ne  is  the  N-version  software  composec  of 
tnree  inceoencenc  versions.  Tne  probability  of  a  system  e-ror  in  t.nis  2- version 
sortware  system  becomes  the  probability  of  at  least  two  versions  executing 
1  r, c 0 rrec 1 1  y .  Inis  simple  case  is  computed  with  Scott's  N-version  prooramming 
rel 1 aci 1 i ty  model  as 


R  =  1  -  [Type  1  Error  +  Type  2  Error  +  Type  3  Error], 

In  Scott's  analysis,  he  assumes  that  the  probability  of  a  Type  3  Error  is 
zero.  Therefore,  with  Scott's  model, 

R  ■  1  "  {  [P(Ii)P(I2)P(l3)]  +  [P(Ci)P( 1 2 ) P ( I3)  + 

P(  I  i)P(C2)P( I3)  +  P( I l)P( l2)p(C3)]  +  0}. 

By  substitution, 


1  -  ( [1  -  P(Ci)][l  -  P(C2)][1 
P(Ci)[l  -  P(C2)][1  -  P(c3)]  + 
[1  -  P(C1]P(C2)[1  -  P(C3)]  + 
[1  -  P  ( C ! )  ]  [  1  -  P  ( C  2 )  ]  P  ( C  3 )  ;. 


-  P(C3)]  ♦ 


Multiplying  out  the  factors  gives 


1  -  !  [1  -  P(Ci)  -  P(C2)  +  P(Ci)P(C2)][l 
P( C ! )  [  1  -  P(C2)  -  P(C3)  +  P( C2)P( C3) ]  + 
P(C2)[1  -  P( C ! )  -  P(C3)  ♦  P ( C j ) P ( C3 ) ]  «• 
P(C3)[1  -  P(C ! )  -  P(C2)  ♦  P(C1)P(C2)]’ 


*  P(C3)] 


Witn  furt.”.ev*  multiplication, 


1  -  [1  -  P(Ci)  -  Pi C2)  ♦  P ( C ! ) P t C 2 )  -  P(C; 
P ( C i ) ? ' C 3 )  +  P(C2)P(C3)  -  P ( C i ) F ( C £ ' P ( C 3 ) 
P(C-J  -  P(C1)P(C2)  -  P(Ci)P(C3)  - 
P(C1)P(C2)P(C3)  -  P ( C 2 )  -  P ( C ]_ ) P ( C 2 )  ' 

P ( C 2 ) P ( c 3 )  +  P(C1)P(C2)P(C3)  +  P ( C 3 j  - 
P  ( C  i )  P  ( C  3 )  -  P  ( C  2 )  P  ( C  3 )  V  P  ( c  x )  P  ( C  2 )  P  ( C  3 } ; 


Cancelling  alike  terns  gives 

GiG2  =  1  -  [1  -  P(C]_)P(C2) 
2?:C1)P(C2)P(C3)j. 


-  P(C1)?(C3)  -  P ( C 2 ) P ( C 3 ) 


This  can  be  reducec  to 


G]G2  «  P(Ci)P(C2)  +  P(CX )P(C3)  +  P(C2)P(C3)  -  2P(Ci)P(C2)P(C3). 


with  Scott's  assumption  that  the  probability  of  a  Type  3  Error  is  zero,  the 
equivalent  assumption  in  the  software  reliability  model  would  be  that 
G2  =  1.  Therefore, 

Si  =  P(C])P(C2)  +  P(C!  )P(C3)  +  P(C2)P(C3)  -  2P(C1)P(C2)P(C3). 
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TECHNICAL  REPORT 

on 

SOFTWARE  RELIABILITY  DATA  AND  DATABASE 

1.0  INTRODUCTION 

7~e  software  reliability  data  recuirec  by  the  software  re! iabi I : ty 
mccel  was  cessi-icec  in  t.ne  previous  section.  Inis  section  cescribes  the 
cata  to  be  collected  by  avionics  systems  deve1ope,-s  prior  to  starting  develop¬ 
ment  of  t.ne  software  packages,  during  tne  envelopment  of  the  software  packages, 
and  during  the  operational  life  of  the  software.  In  addition,  attributes 
of  the  data  base  program  are  described. 

2.0  BACKGROUND 

As  noted  in  the  previous  section  of  this  report,  the  software  reli¬ 
ability  model's  primary  inputs  are  probabilities.  While  this  is  an  excellent 
form  for  the  model,  it  is  not  the  normal  type  of  data  collected  by  avionics 
systems  developers.  The  data  collected  by  developers  of  avioncs  systems 
tends  to  be  deterministic  as  opposed  to  statistical.  The  statistical  and 
probabilistic  values  are  derived  from  the  determi nsti c  data  as  discussed 
in  earlier  sections  of  this  report.  The  method  used  by  Battelle  to  derive 
these  data  ties  in  closely  with  the  database  program. 

Although  any  database  manager  can  store  anc  retrieve  information, 
spreadsheet  programs  provide  more  caoability  than  a  simple  file  manager. 

The  spreadsheet  programs  with  their  built  in  functions  provide  the  capability 
to  analyze  the  data  instead  of  simply  storing  and  retrieving  data.  There 
are  a  large  number  of  spreadsheet  programs  to  choose  from  and  the  selection 
of  a  specific  program  is  not  within  the  scope  of  this  work.  This  section 
does  discuss  factors  which  should  be  considered  in  the  selection  of  a  spread¬ 
sheet  program. 
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3.0  DATABASE  PROGRAM  AND  SOFTWARE  RELIABILITY  DATA 

The  data  to  be  collected  by  the  avionics  system  develoDe’"  is  collected 
at  different  times  curing  the  software  life  cycle.  It  is  likely  to  be  organized 
in  any  catabase  program  as  an  assortment  of  different  files.  In  orce;-  to 
maniou’ate  these  cata  contained  in  different  files  into  the  form  recuirec 
by  tne  software  reliability  model,  a  relational  database  manage1"  wnicn  can 
l’nk  secarate  data  files  to  create  a  dataoase  containing  information  selectee 
from  tne  different  files  is  recommended. 

Principles  c*  data  design  should  be  aooliec  wnen  developing  the 
Catabase.  Data  cesign  is  a  set  of  principles  anc  analytic  tools  that  brings 
to  the  cesign  of  data  tne  same  kind  of  organization  tnat  structured  programs 
brings  to  programs.  To  avoid  dangerous  file  designs,  a  clear  understanding 
of  the  dependencies  in  the  files  is  necessary.  Transitive  depencencies  are 
a  frequent  cause  of  structural  problems  in  the  design  of  files. 

Once  the  database  is  created,  use  of  a  spreadsheet  with  its  built-in 
arithmetic  and  statistical  functions  will  provide  the  capability  to  analyze 
deterministic  data  such  as  lines  of  source  code,  memory  usage,  errors  detected, 
errors  corrected,  and  linkages  and  assemble  these  data  into  the  statistical 
form  required  for  determining  the  probabilistic  inputs  required  by  the  software 
reliability  model . 

The  inputs  required  for  the  database  program  will  depend  upon  the 
software  reliability  model  (reference  Section  2,  "Technical  Report  on  Review 
of  Previous  Studies  of  Software  Reliability  Models"  for  some  examples)  used 
to  obtain  the  probabi 1 i sti c  reliability  value  that  is  required  in  the  software 
reliability  model  described  in  Section  4,  "Technical  Report  on  Formulation 
of  the  Software  Reliability  Model".  Table  1  lists  some  of  the  software  reliabili 
models  and  their  required  inputs.  The  built-in  arithmetic  and  statistical 
functions  will  manipulate  this  input  data  to  obtain  the  probabilistic  reli¬ 
ability  values.  An  example  of  what  the  input  and  output  for  the  database 
program  might  look  like  is  given  in  Tables  2  and  3. 
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Table  1.  Input  Data  Used  by  Various  Software  Reliability  Models 


Software 

Rel 1 ab i 1 i ty  Model 


Input  Data 


Generalized  Imperfect 

Debugging  Model 

N  =  the  total  number  of  errors; 
p  =  the  probability  of  perfect 
programmer  debugging  behavior 

3ug-?rooortional  Mocel 

c  r( t )  =  the  number  of  remaining 
bugs 

Geometric  Poisson  Model 

x  =  the  average  number  of  faults 
occurring  in  the  first 
i nterval ; 

t-j  =  the  i-th  debugging  interval 

Schneidewind  Non-Homogeneous 
Poisson  Model 

m-j  =  the  estimated  number  of 
errors  in  interval  i 

Jel inski -Moranda 

De-Eutrophication  Model 

N  =  the  number  of  initial  errors 
in  the  program; 

Xi  =  the  length  of  the  i-th 
debugging  interval  ; 
n  =  the  number  of  errors  found 
to  date 

Extended  Jel inski-Moranda 

Model 

N  =  the  total  number  of  initial 
errors ; 

n-j  =  the  cumulative  number  of 
errors  found  through  the 
i-th  interval  ; 

tj  *  the  i-th  debugging  interval 

Geometric  De-Eutrophi cati on 

Model 

D  =  the  initial  errcr  detection 
rate ; 

n  =  the  total  number  of  errors 
discovered ; 

X-j  =  the  i-th  debugging  interval 

Lines  of  Source  Code  Model 

n  =  the  total  numoer  of  lines  of 
source  code;  the  programming 
language  (i.e.,  FORTRAN, 

Cobol  ,  Ada ,  etc. ) 

r. 


Mooel 

Geometric  Poisson 

Non-Homocenecus 
Poi sson 


Geometric  Poisson 

Non-Homoceneous 
Poi sscn 


Geometric  Poisson 

Non-Homogeneous 
Poi sson 

Generalized  Poisson 


estimated 

N 


Observec 

Errors 


Estimateo 


0.0055 

0.0056 


0.1556 

0.1727 


Generalized  Poisson 

28 

23 

0.2072 

IBM  Poisson  (Modified) 

37 

23 

0.0082 

Geometric  Poisson 

155 

73 

0.0204 

Non-Homogeneous 

Poisson 

155 

I 

73 

1 

0.0206 
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